Drug adverse event extraction method and apparatus

ABSTRACT

A method of extracting a combination of a drug and an adverse event related to the drug includes: for each of positive example combinations, negative example combinations and combinations that are neither positive examples nor negative examples, which are combinations of drug and disease, extracting medical events from medical information data about a patient and generating attribute data based on time-series information about the medical events; and learning a discriminant model based on attribute data of the positive and negative examples; and inputting attribute data corresponding to the combinations that are neither positive examples nor negative examples to the discriminant model to determine scores.

TECHNICAL FIELD

The present invention relates to a method and apparatus for extractingadverse events caused by medical drugs by information processing, and inparticular to a drug adverse event extraction method and apparatuscapable of widely extracting unknown adverse events caused by drugs.

BACKGROUND ART

Though drugs appear on the market after being approved by thegovernment, there may be a case where, when a drug appears on the marketand is prescribed to many patients, an unexpected drug adverse eventoccurs and brings about serious health damage. This is because, sinceclinical trials performed to gain approval by the government areperformed for a limited number of patients in a short period in order toverify the effectiveness of a drug, it is difficult to detect all drugadverse events of the drug by the clinical trials. Therefore, it is animportant role of a drug regulatory authority to conduct post-marketingsurveillance of drugs on the market to detect drug adverse events thathave not been found yet early and carry out safety measures to preventoccurrence of the drug adverse events.

In recent post-marketing surveillance, detection of drug adverse eventsis performed mainly by analyzing spontaneous reports. Spontaneousreports are reports about events suspected to be drug adverse eventsthat are spontaneously provided by doctors, patients and pharmaceuticalcompanies. However, since all drug adverse events that have actuallyoccurred at clinical sites are not necessarily reported in spontaneousreports, there is a disadvantage that, even if a drug adverse event isdetected from spontaneous reports, it is difficult to detect all drugadverse events that have not been found yet.

In order to make up for this disadvantage, attempts to analyze medicalinformation data, which is information about medical services patientshave received, to extract information about unknown drug adverse eventshave been considered recently. In the medical information data,histories of medical examinations for a huge number of patients thathave actually occurred at clinical sites for a long period is describedin detail unlike spontaneous reports. Therefore, it is expected thatdrug adverse events that have not been reported by spontaneous reportscan be detected by analyzing the medical information data. The medicalinformation data is data obtained from itemized statements of medicalfee and itemized statements of dispensing fee, data obtained frommedical examination records and the like. The itemized statements ofmedical fee and itemized statements of dispensing fee are also referredto as health insurance claims, and the medical examination records arealso referred to as patients' charts or medical records.

Especially, when a patient receives a medical examination at a medicalinstitution using a medical insurance system or a health insurancesystem by the government or a private enterprise, the itemized statementof medical fee and the itemized statement of dispensing fee arecollected to operators of the medical insurance and the healthinsurance. Therefore, it is expected that, by analyzing a huge number ofitemized statements of medical fee and itemized statements ofexamination fee collected to the medical and health insurance operatorsand those entrusted with the insurances, unknown drug adverse events canbe detected.

As related art of the present invention, [NPL1] discloses a method ofacquiring time-series data showing, for each patient, which drug wasprescribed for the patient, and when and which disease occurred in thepatient, and extracting drug adverse events by machine learning based onthe acquired time-series data. In the method of [NPL1], a combination of“drug and disease” which is already known as a combination indicating adrug adverse event is regarded as a positive example, and a combinationof “drug and disease” which is already known as a combination notindicating a drug adverse event is regarded as a negative example. Forexample, if a certain drug is an antipyretic, and it is known thateruption occurs as an adverse event when the antipyretic is taken, then,the combination of “the drug and eruption” is a positive example. If theantipyretic is prescribed because fever is observed in the patient, thecombination of “the antipyretic and fever” is classified as a negativeexample because the antipyretic itself is a drug for coping with andlowering the fever.

In the method of [NPL1], with positive examples, negative examples,combinations of “drug and disease” which are neither positive examplesnor negative examples, and the time-series data of prescription of drugsand occurrence of diseases used as input, attribute data showing whenand how many times a disease occurred during a drug prescription periodis created for each combination of “drug and disease”, and a model forcalculating a score indicating the degree of suspicion that acombination of “drug and disease” is an adverse event from the attributedata based on attribute data corresponding to the positive examples andattribute data corresponding to the negative examples. Hereinafter, thismodel will be called “a discriminant model”. Attribute datacorresponding to the combinations of “drug and disease” which areneither positive examples nor negative examples is inputted to thelearned discriminant model to calculate the above score for each of thecombinations of “drug and disease”. Since this score indicates thedegree of possibility of the inputted combination of “drug and disease”which is neither a positive example nor a negative example being a drugadverse event, combinations of “drug and disease” suspected to be drugadverse events are extracted based on calculated scores.

The technique described in [NPL1] is basically a technique in which, oninputted time-series data of drugs and diseases, attention is paid onlyto occurrence of the diseases during prescription periods of the drugs,and combinations of “drug and disease” indicating adverse events areextracted from time-series information about prescription of the drugsand time-series information about the observed diseases.

CITATION LIST Non Patent Literature(s)

-   [NPL1] “OMOP Cup Grand Prize ‘Best Submission’ Report”, Foundation    for the National Institute of Health, [retrieved on Jan. 18, 2014],    the Internet <URL:    http://omop.fnih.org/sites/default/files/Vogel_Progress_prize_methods_GP_0.pdf>-   [NPL2] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J.    Lin, “LIBLINEAR: A library for large linear classification,” Journal    of Machine Learning Research, 9(2008), 1871-1874.-   [NPL3] J. Zhou, J. Chen and J. Ye, “MALSAR: Multi-tAsk Learning via    Structural Regularization,” Arizona State University, 2012,    [retrieved on Feb. 26, 2014], the Internet <URL:    http://www.public.asu.edu/˜jye02/Software/MALSAR>-   [NPL4] Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W.    P., “Smote: Synthetic minority over-sampling technique,” Journal of    Artificial Intelligence Research, 16:321-357 (2002).-   [NPL5] Xu-Ying Liu and Jianxin Wu and Zhi-Hua Zhou, “Exploratory    Under-Sampling for Class-Imbalance Learning,” Data Mining, 2006.    ICDM '06. Sixth International Conference on.-   [NPL6] Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, H., von    Buenau, P., and Kawanabe, M, “Direct importance estimation for    covariate shift adaptation,” Annals of the Institute of Statistical    Mathematics, vol. 60, no. 4, pp. 699-746, 2008.

SUMMARY OF INVENTION Technical Problem

On the practical use for drug safety measures, it is required to extractdrug adverse events from medical information data with a high accuracy.Here, the high accuracy refers to extraction of broad kinds of adverseevents with few mistakes. This is because it is important to grasp broadkinds of adverse events accurately in order to prevent health damage dueto adverse events.

In order to derive a rule for discriminating between a positive exampleand a negative example as in the technique described in [NPL1], it isnecessary that there is some difference in attribute data used forlearning a discriminant model between positive examples and negativeexamples. If the difference does not exist, it is not possible to, whenattempting to perform classification into positive examples and negativeexamples by allocating a high score to the positive examples and a lowscore to negative examples, calculate such scores that distinguishbetween the positive examples and the negative examples.

In the technique described in [NPL1], attribute data showing theoccurrence time and occurrence frequency of a disease during aprescription period of a drug is created for each of positive examplesand negative examples, and the attribute data is inputted. In theattribute data used in the [NPL1], however, a difference between thecontent of attribute data of a combination indicating an adverse event(a positive example combination) and the content of attribute data of acombination which is not an adverse event (a negative examplecombination) may be small In such a case, it is difficult todiscriminate between a combination indicating an adverse event and acombination which is not an adverse event only from the attribute datacreated from the time-series information about drugs and diseases.Therefore, the technique described in [NPL1] has a problem that it isnot possible to broadly extract only combinations indicating adverseevents with few mistakes.

Hereinafter, description will be made on an example in which thedifference in attribute data is small between a combination indicatingan adverse event and a combination which is not an adverse event.

The combination which is not an adverse event shows a disease which ishardly thought to be caused by prescription of a drug. As an examplethereof, a combination of “drug and disease”, which are a certaindisease and a drug prescribed for the purpose of treatment of thedisease, is given. As for this combination, since the drug is prescribedfor the purpose of treatment of the disease when the disease occurs, thenumber of times that prescription of the drug and occurrence of thedisease occur on the same day is large on time-series information aboutdrugs and diseases. On the other hand, among adverse events, there issuch an adverse event that a symptom appears immediately after a drug isprescribed, such as an allergic reaction. As for a disease indicatingsuch an adverse event also, the number of times that the disease occurson the same day when a drug is prescribed is large on the time-seriesinformation about drugs and diseases. Therefore, there may be a casewhere the difference in the content of the attribute data showing whenand how many times disease occurred after prescription of a drug issmall between a combination indicating an adverse event and acombination which is not an adverse event. In this case, it is notpossible to distinguish between the combination indicating an adverseevent and the combination which is not an adverse event even if adiscriminant model is used.

Thus, an object of the present invention is to solve the problem of therelated art and provide a drug adverse event extraction method andapparatus capable of accurately extracting a combination of a drug andan adverse event related to the drug.

Solution to Problem

A drug adverse event extraction method of the present invention is adrug adverse event extraction method of extracting a combination of adrug and a disease corresponding to a drug adverse event, the methodcomprising, on the assumption that combinations already known ascombinations indicating drug adverse events are regarded as positiveexample combinations, combinations already known as combinations notbeing drug adverse events are regarded as negative example combinations,and given combinations being neither positive example combinations nornegative example combinations are regarded as combinations other thanpositive and negative examples:

generating, using medical information data that includes time-seriesinformation about medical events for each patient, attribute data foreach of the positive example combinations, for each of the negativeexample combinations and for each of the combinations other thanpositive and negative examples, based on the time-series informationabout the medical events;

learning a discriminant model by the attribute data corresponding to thepositive example combinations and the attribute data corresponding tothe negative example combinations;

inputting the attribute data corresponding to the combinations otherthan positive and negative examples to the discriminant model tocalculate scores; and

applying an extraction condition to the score calculated for each of thecombinations other than positive and negative examples to extractcombinations other than positive and negative examples being suspectedto be drug adverse events,

wherein the medical events for each patient include prescription of adrug for the patient and a disease observed in the patient, and

wherein the medical events for each patient further include at least oneof a medical act performed for the patient and an event showing that themedical act has been performed accompanying the medical act performedfor the patient.

A drug adverse event extraction apparatus of the present invention is adrug adverse event extraction apparatus for extracting a combination ofa drug and a disease corresponding to a drug adverse event, theapparatus comprising, on the assumption that combinations already knownas combinations indicating drug adverse events are regarded as positiveexample combinations, combinations already known as combinations notbeing drug adverse events are regarded as negative example combinations,and given combinations being neither positive example combinations nornegative example combinations are regarded as combinations other thanpositive and negative examples:

attribute creation means that generates, using medical information datathat includes time-series information about medical events for eachpatient stored in a storage device, attribute data for each of thepositive example combinations stored in the storage device, for each ofthe negative example combinations stored in the storage device and foreach of the combinations other than positive and negative examplesstored in the storage device, based on the time-series information aboutthe medical events, and stores the attribute data into the storagedevice;

learning means that learns a discriminant model by the attribute datacorresponding to the positive example combinations and the attributedata corresponding to the negative example combinations;

calculation means that inputs the attribute data corresponding to thecombinations other than positive and negative examples stored in thestorage device to the discriminant model to calculate scores; and

extraction means that applies an extraction condition to the scorecalculated for each of the combinations other than positive and negativeexamples to extract combinations other than positive and negativeexamples being suspected to be drug adverse events,

wherein the medical events for each patient include prescription of adrug for the patient and a disease observed in the patient, and

wherein the medical events for each patient further include at least oneof a medical act performed for the patient and an event showing that themedical act has been performed accompanying the medical act performedfor the patient.

According to the present invention, it becomes possible to accuratelyextract a combination of a drug and an adverse event related to thedrug.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration of a drug adverseevent extraction apparatus of an exemplary embodiment;

FIG. 2 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 1;

FIG. 3 is a diagram showing a relationship between a first period and asecond period;

FIG. 4 is a diagram showing relationships among first to fourth periods;

FIG. 5 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 1;

FIG. 6 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 5;

FIG. 7 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 2;

FIG. 8 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 7;

FIG. 9 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 3;

FIG. 10 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 9;

FIG. 11 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 4;

FIG. 12 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 11;

FIG. 13 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 5;

FIG. 14 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 13;

FIG. 15 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 6;

FIG. 16 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 15;

FIG. 17 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 7;

FIG. 18 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 17;

FIG. 19 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 8;

FIG. 20 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 19;

FIG. 21 is a block diagram showing a configuration of a drug adverseevent extraction apparatus according to Modification 9;

FIG. 22 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 21;

FIG. 23 is a block diagram showing a configuration of a drug adverseevent extraction apparatus of according to Modification 10; and

FIG. 24 is a flowchart showing the operation of the drug adverse eventextraction apparatus shown in FIG. 23.

DESCRIPTION OF EMBODIMENTS

Next, exemplary embodiments will be described with reference todrawings.

First, time-series information about medical events will be described.

Medical information data such as an itemized statement of medical fee,an itemized statement of dispensing fee and medical examination recordinformation can be regarded as time-series information about medicalevents for each patient because the medical information data includesrecord about medical treatment for each day for the patient. Here, amedical event refers to a medical-related event which occurs on aparticular patient at a particular time point, such as a certain patienthaving a disease observed, having a particular drug prescribed,receiving a particular medical act or receiving a diagnosis of aparticular disease at a certain time point. Here, more generally,information related to medical services in a broad sense, such aswritten matters about a particular item at a particular time point for aparticular patient, medical expenses, a hospital department andhospitalization, which is included in the medical information data isalso referred to as a medical event. Therefore, in the presentspecification, a medical event is defined as such that includes not onlya disease which occurs in a patient but also a medical act performed forthe patient and an event showing that the medical act was performedaccompanying the medical act performed for the patient. It goes withoutsaying that the medical event “hospitalization” indicates that aparticular patient was hospitalized at a particular time point; themedical event “medical expenses” indicates that a particular amount ofmedical expenses at a particular time point for a particular patient ischarged; and the medical event “hospital department” indicates ahospital department at which a particular patient had a medicalexamination at a particular time point. For example, measures such as atreatment act at a particular hospital department, prescription of adrug and hospitalization are included in medical acts, and request formedical expenses is given as an event showing that a medical act wasperformed accompanying the medical act.

Further, the time-series information about medical events is assumed tobe such that whether particular drug A was prescribed for certainpatient X is expressed in appropriately separated time units (forexample, units of one month), for example, as shown below.

Patient X; Drug A: 0, 0, 0, 1, 1, 0, 0

Here, occurrence of the event is indicated by “1”, and non-occurrence isindicated by “0”. In this example, it is shown that prescription of drugA for patient X is not performed for the first three months (that is,the first to third months), performed for the following two months (thefourth and fifth months), and not performed for the further followingtwo months (the sixth and seventh months).

Though whether prescription of a drug occurs or not is expressed bybinary data of “0” and “1” in the above example, the expression is notlimited to binary data. It is also possible to express the amount ofprescription in an appropriate unit to express information in detail asshown below:

Patient X; Drug A: 0, 0, 0, 2.3, 6.18, 0, 0

Further, though a medical event indicating prescription of a drug isshown in the above example, it is also possible to give informationabout a hospital department at which each medical drug was prescribed toexpress the medical event in detail as shown below:

Patient X; Drug A; Internal medicine department: 0, 0, 0, 1, 1, 0, 0

Patient X; Drug A; Surgery department: 0, 0, 0, 1, 0, 0, 0

Further, one of the kinds of medical events included in the medicalinformation data is medical expenses. This event indicates that aparticular amount of medical expenses at a particular time point for aparticular patient is charged. As an example of time-series informationabout the medical event of medical expenses, how much is charged formedical expenses for certain patient X is expressed by a billing amountin an appropriate unit (for example, medical fee point) as shown below.

Patient X; Medical expenses: 300, 550, 90, 140, 2500, 600, 0

Medical information data includes time-series information about aplurality of kinds of medical events for an enormous number of patients.For example, the following example shows time-series information about aplurality of kinds of medical events for two patients.

Patient X; Drug A: 0, 0, 0, 1, 1, 0, 0

Patient X; Drug B: 0, 0, 0, 0, 0, 1, 0

Patient X; Medical act C: 0, 0, 1, 1, 1, 0, 0

Patient X; Diagnosed disease name D: 0, 0, 0, 0, 1, 1, 0

Patient X; Hospitalization: 0, 0, 0, 0, 1, 0, 0

Patient X; Internal medicine department (hospital department): 0, 0, 1,1, 1, 1, 0

Patient Y; Drug A: 0, 0, 1, 1, 1, 1, 1

Patient Y; Drug C: 0, 0, 0, 1, 1, 1, 0

The above example shows, for drugs A and B having been prescribed forpatient X, medical act C having been performed for patient X, diseasename D having been diagnosed for patient X, patient X having beenhospitalized, and patient X having had a medical examination at aninternal medicine department, when each occurred. It is also shown whenprescription of drugs A and C occurred for patient Y.

Hereinafter, for simplification, the medical event “diagnosed diseasename” is referred to as “disease”. When “diagnosed disease name” isreplaced with “disease” in the above example, the following result isobtained.

Patient X; Drug A: 0, 0, 0, 1, 1, 0, 0

Patient X; Drug B: 0, 0, 0, 0, 0, 1, 0

Patient X; Medical act C: 0, 0, 1, 1, 1, 0, 0

Patient X; Disease D: 0, 0, 0, 0, 1, 1, 0

Patient X; Hospitalization: 0, 0, 0, 0, 1, 0, 0

Patient X; Internal medicine department (hospital department): 0, 0, 1,1, 1, 1, 0

Patient Y; Drug A: 0, 0, 1, 1, 1, 1, 1

Patient Y; Drug C: 0, 0, 0, 1, 1, 1, 0

FIG. 1 shows a configuration of a drug adverse event extractionapparatus of an exemplary embodiment. This drug adverse event extractionapparatus has a function of extracting a combination indicating anadverse event from among combinations of “drug and disease” based ontime-series information about medical events for a large number ofpatients included in medical information data. Especially, thisapparatus extracts a combination indicating an adverse event based on ascore calculated for each combination, that is, a numerical valueindicating suspicion as an adverse event. In the description below, thecombination of “drug and disease” will be referred to simply as a“combination” as far as it is apparent from the context.

As shown in FIG. 1, the drug adverse event extraction apparatus isprovided with processing apparatus 11 that executes data processing;storage device 12 that is connected to processing apparatus 11 and thatis for storing medical information data to be a target of extraction ofdrug adverse events, and for storing data required for data processingat processing apparatus 11 and generated as a result of data processing,such as a discriminant model, scores and an extraction result;communication interface (I/F unit) 13; operation input unit 14; andscreen display unit 15. All of communication interface unit 13,operation input unit 14 and screen display unit 15 are connected toprocessing apparatus 11.

Communication interface unit 13 is configured with a dedicated datacommunication circuit and has a function of performing datacommunication between various kinds of apparatuses not shown andprocessing apparatus 11 that are connected via a communication circuit.Operation input unit 14 is configured with operation input devices suchas a keyboard and a mouse and has a function of detecting an operator'soperation and performing output to processing apparatus 11. Screendisplay unit 15 is configured with a screen display apparatus such as anLCD (liquid crystal display) and a PDP (plasma display panel) and has afunction of screen-displaying various kinds of information, such as anoperation menu and a selection result, in response to an instructionfrom processing apparatus 11.

Storage device 12 is configured with a hard disk, a semiconductor memorydevice or the like. In the drug adverse event extraction apparatus shownin FIG. 1, main information stored in storage device 12 are: medicalinformation data 51, positive example combinations 52, negative examplecombinations 53, combinations other than positive and negative examples54, positive/negative example flags 55, attribute data 56, discriminantmodel 57, adverse event scores 58, extraction condition 59, extractionresult 60 and control parameters 69. In addition to these, informationused for the operation of the drug adverse event extraction apparatusmay be stored in storage device 12. The above information will bedescribed below.

Medical information data 51 is information obtained from itemizedstatements of medical fee, itemized statements of dispensing fee,medical record and the like as described above, and this is expressed astime-series information about medical events for each patient. In thepresent exemplary embodiment, medical information data 51 is configuredwith medical events that occurred at particular time points forparticular patients, including a certain patient (a) having a particulardrug prescribed, (b) having a particular disease observed, (c) receivinga particular medical act, (d) being charged for a particular amount ofmedical expenses, (e) having a medication examination at a particularhospital department and (f) being hospitalized, at certain points oftime. Though the details of the medical events will be described later,items other than those given here can be used as medical events in thepresent invention if the items are related to medical services in abroad sense. Further, in the present invention, it is also possible notto use a part of the medical events (c) to (f) described here dependingon how the combination of “drugs and disease” to be a drug adverseevents is found out. As described in relation to [NPL1], it is notpossible to accurately extract a drug adverse event from time-seriesdata of prescription of drugs and occurrence of diseases. Therefore, inthe present exemplary embodiment, attention is paid not only to adisease but also to medical events other than occurrence of the diseaseduring a prescription period of a drug. In the present exemplaryembodiment, it is preferable that the medical information data isinformation obtained from either itemized statements of medical fee oritemized statements of dispensing fee.

The present invention is for making it possible to discriminate whethera result of an adverse event was caused by prescription of a drug.Therefore, examples of combining an event to be a cause (a precedingevent, that is, prescription of a drug) and an event possibly to be aresult (a succeeding event, that is, an observed disease) areconsidered, and, for each combination, whether the event is an adverseevent related to the drug is considered. Thus, in the present exemplaryembodiment, positive example combinations 52, negative examplecombinations 53 and combinations other than positive and negativeexamples 54, all of which are combinations of “drug and disease” areassumed. Positive example combinations 52 are combinations of “drug anddisease” which are already known as combinations indicating drug adverseevents. Negative example combinations 53 are combinations of “drug anddisease” which are already known as combinations that are not drugadverse events. In comparison, combinations other than positive andnegative examples 54 means such that are combinations of “drug anddisease” but are neither positive example combinations nor negativeexample combinations. Therefore, combinations other than positive andnegative examples 54 are combinations that are known neither ascombinations indicating drug adverse events nor as combinations that arenot drug adverse events.

Positive/negative example flags 55 are flag values according tocombinations, which indicate whether a positive/negative examplecombination is a positive example combination or a negative examplecombination. As the flag values, for example, a value indicating apositive example is set for a positive example combination, and a valueindicating a negative example is set for a negative example combination.

Attribute data 56 is data showing, for each of the positive examples,the negative examples and the combinations other than positive andnegative examples, characteristics on the medical information data. Thedetails of the attribute data in the present exemplary embodiment willbe described later.

Discriminant model 57 is a model that shows a relationship betweenattribute data corresponding to a combination and whether thecombination corresponds to an adverse event or not. As a form ofdiscriminant model 57, for example, a logistic regression model, alinear support vector machine (SVM) model and the like are conceivable.

Adverse event score 58 is a value indicating suspicion as an adverseevent that is calculated for each of combinations other than positiveand negative examples 54 by discriminant model 57. The larger the valueis, the stronger the suspicion as an adverse event is.

Extraction condition 59 shows a condition to be satisfied at the time ofextracting a combination indicating an adverse event from amongcombinations other than positive and negative examples 54. Examples ofthe extraction condition include a threshold for adverse event scores ofcombinations to be extracted, the maximum number of combinations to beextracted, and the like.

Extraction result 60 is a list of combinations extracted from amongcombinations other than positive and negative examples 54, ascombinations indicating adverse events.

Control parameters 69 are various kinds of parameters specifying anexecution condition for a drug adverse event extraction process and thelike in processing apparatus 11.

Next, processing apparatus 11 will be described.

The drug adverse event extraction apparatus of the present exemplaryembodiment learns discriminant model 57 by positive example combinations52 and negative example combinations 53 and, after that, appliescombinations other than positive and negative examples 54 todiscriminant model 57 to obtain adverse event scores 58 for combinationsother than positive and negative examples 54. In order to execute such aprocess, processing apparatus 11 is provided with input unit 21,attribute data creation unit 22, discriminant model learning unit 23,adverse event score calculation unit 24 and extraction unit 25.Attribute data creation unit 22 corresponds to attribute creating means;discriminant model learning unit 23 corresponds to learning means;adverse event score calculation unit 24 corresponds to calculationmeans; and extraction unit 25 corresponds to extraction means.

Input unit 21 inputs information required for the process in the drugadverse event extraction apparatus, such as medical information data,positive example combinations, negative example combinations,combinations other than positive and negative examples and an extractioncondition, from communication interface unit 13 or operation input unit14 and stores the information into storage device 12. Here, the medicalinformation data given to input unit 12 is assumed to be time-seriesinformation about medical events for each patient extracted fromitemized statements of medical fee, itemized statements of dispensingfee and the like. Recently, itemized statements of medical fee anditemized statements of dispensing fee are created as electronic data ina pre-defined data format, and the data format itself shows time-seriesinformation about medical events. Therefore, it is extremely easy toextract the time-series information about medical events for eachpatient from itemized statements of medical fee and itemized statementsof dispensing fee.

Attribute data creation unit 22 reads positive example combinations 52,negative example combinations 53, combinations other than positive andnegative examples 54 and medical information data 51 from storage device12 and performs preprocessing for medical information data 51 which hasbeen read. After that, attribute data creation unit 22 creates attributedata using the read information and stores the attribute data intostorage device 12. The preprocessing is not necessarily to be performed,depending on the data format of the medical information data and thelike.

Discriminant model learning unit 23 has a function of: reading positiveexample combinations 52 and negative example combinations 53, theattribute data corresponding to the positive examples and the negativeexamples among attribute data 56 and positive/negative example flags 55from storage device 12; learning discriminant model 57; and storinglearned discriminant model 57 into storage device 12.

Adverse event score calculation unit 24 has a function of: readingcombinations other than positive and negative examples 54 and theattribute data corresponding to the combinations other than positive andnegative examples from storage device 12; inputting the read attributedata to the discriminant model to calculate an adverse event score foreach of the combinations other than positive and negative examples; andstoring the calculated adverse event score into storage device 12.

Extraction unit 25 has a function of: reading adverse event scores 58and extraction condition 59 from storage device 12; extracting acombination suspected to be an adverse event, from among thecombinations other than positive and negative examples in a manner thatthe extraction condition is satisfied; and storing a result of theextraction into storage unit 12. Further, extraction unit 25 also has afunction of outputting the extraction result to screen display unit 15or to the outside via communication interface unit 13.

Next, the operation of the drug adverse event extraction apparatus shownin FIG. 1 will be described with reference to FIG. 2. The operation ofthe drug adverse event extraction apparatus is roughly divided in fourphases of attribute data creation phase S1, learning phase S2, adverseevent score calculation phase S3 and extraction phase S4, and the phasesare executed in that order.

At attribute data creation phase S1, input unit 21 receives medicalinformation data, each combination of positive examples, negativeexamples, combinations other than positive and negative examples fromcommunication interface unit 13 or operation input unit 14 and storesthem into storage device 12 at step S11. Next, at step S12, attributedata creation unit 22 reads out medical information data 51, positiveexample combination 42, negative example combinations 53 andcombinations other than positive and negative examples 54 from storagedevice 12, performs preprocessing for medical information data 51. Afterthat, at step S13, attribute data creation unit 22 creates attributedata corresponding to each of the read-out combinations and stores thecreated attribute data into device apparatus 12.

The attribute data is data showing, for a combination, characteristicsof occurrence and non-occurrence of other medical events at a time closeto a time point when the drug and the disease of the combinationco-occur on the same patient, on inputted time-series information aboutmedical events.

At learning phase S2, discriminant model learning unit 23 calls positiveexample combinations 52, negative example combinations 53, attributedata 56 corresponding to the positive and negative examples,positive/negative example flags 55 and discriminant model 57 fromstorage device 12 at step S21, and learns the discriminant model usingthese at step S22. The learned discriminant model is returned to storagedevice 12.

At adverse event score calculation phase S3, adverse event scorecalculation unit 24 reads out discriminant model 57, combinations otherthan positive and negative examples 54 and the attribute datacorresponding to the combinations from storage device 12 at step S31,and applies the read-out attribute data to the discriminant model tocalculate adverse event scores at step S32. The calculated adverse eventscores are stored into storage device 12.

At extraction phase S4, input unit 21 receives an extraction conditionfrom communication interface unit 13 or operation input unit 14 andstores the extraction condition into storage device 12 at step S41first. Next, at step S42, extraction unit 25 reads out adverse eventscores 58 and extraction condition 59 from storage device 12, extractscombinations indicating adverse events from among the combinations otherthan positive and negative examples in a manner that the extractioncondition is satisfied, and stores a result of the extraction intostorage device 12. After that, at step S43, extraction unit 25 outputsthe extraction result to screen display unit 15 or to the outside viacommunication interface unit 13. At this time, it is preferable topreferentially extract a combination with a high adverse event score,which is strongly discriminated as an adverse event, and prevent acombination with a low adverse event score from being preferentiallyextracted. That is, it is preferable to sort and output the combinationsother than positive and negative examples in descending order of thelevel of being suspected to be an adverse event. Further, it is possibleto input a lot of combinations of “drug and disease” for various drugsin the present exemplary embodiment, and, in this case, it is preferableto furthermore output a result of sorting in order of adverse eventscores for each kind of drug because it is convenient to knowcombinations suspected to be adverse events for each drug.

Next, operations of attribute data creation phase S1, learning phase S2,adverse event score calculation phase S3 and extraction phase S4 will bedescribed in more detail.

(1) Details of attribute data creation phase S1

(1-1) Step S11

At step S11, medical information data 51, combinations of positiveexamples and negative examples, and combinations other than positive andnegative examples (that is, positive example combinations 52, negativeexample combinations 53 and combinations other than positive andnegative examples 54) and a period length condition are read out fromstorage device 12. Examples of the positive examples, the negativeexamples, and the combinations other than positive and negative exampleswill be shown below. At attribute data creation phase S1, allcombinations are treated equally without being conscious of whether eachcombination is a positive example, a negative example or a combinationother than positive and negative examples. The period length conditionis stored in storage device 12 in advance as one of control parameters69.

Examples of combinations read out from storage device 12 are shownbelow.

(Drug A; Disease C), (Drug B; Disease B), (Drug C; Disease A), . . . .

Further, an example of inputted medical information data 51 is shownbelow. It is assumed that inputted medical information data is the datashown below. Though the time unit is a month in the example, the timeunit may be a day, a week or a year. Hereinafter, for simplification,description will be made on the case where the time unit is a month.

Patient X, Drug A; Internal medicine department: 0, 0, 1, 0, 0, 0, 0

Patient X, Drug B; Internal medicine department: 1, 0, 1, 0, 0, 0, 0

Patient X; Disease A: 0, 0, 0, 0, 1, 0, 0

Patient X; Medical act C: 1, 0, 1, 1, 1, 0, 0

Patient X; Medical act D: 0, 0, 1, 1, 1, 0, 0

Patient X; Hospitalization: 0, 1, 0, 1, 1, 0, 0

Patient X; Internal medicine department (hospital department): 0, 0, 1,1, 1, 0, 0

Patient X, Medical expenses: 300, 550, 90, 140, 2500, 600, 0

Patient Y, Drug A; Internal medicine department: 0, 0, 0, 0, 1, 0, 0

Patient Y, Drug C; Dermatology department: 0, 0, 1, 0, 1, 0, 0

Patient Y; Disease A: 0, 0, 1, 0, 1, 0, 0

Patient X; Disease D: 0, 0, 0, 0, 1, 1, 0

Patient Y; Hospitalization: 0, 0, 0, 0, 1, 0, 0

Patient Y; Internal medicine department (hospital department): 0, 0, 1,0, 1, 1, 0

Patient Y; Dermatology department (hospital department): 0, 0, 0, 0, 1,0, 0

Patient Y; Medical expenses: 0, 0, 190, 0, 1650, 400, 0

Patient Z, Drug B; Internal medicine department: 0, 1, 1, 1, 0, 0, 0

Patient Z; Disease A: 0, 1, 1, 0, 0, 0, 0

Patient Z; Disease D: 0, 0, 0, 1, 0, 0, 1

Patient Z; Medical act C: 0, 1, 1, 1, 0, 0, 1

Patient Z; Hospitalization: 0, 1, 0, 1, 0, 0, 0

Patient Z; Internal medicine department (hospital department): 0, 0, 1,1, 0, 0, 0

Patient Z; Surgery department (hospital department): 0, 0, 1, 1, 0, 0, 1

Patient Z; Medical expenses: 0, 390, 550, 1000, 0, 0, 300

(1-2) Step S12

At step S12, time-series information about new medical events requiredfor creation of attribute data performed at the next step is created byperforming preprocessing described below for medical information data 51read out from storage device 12 at step S11. Furthermore, as for thetime-series information about each medical event included in medicalinformation data 51 read out from storage device 12, since it is notnecessary to treat the time-series information about the medical eventin certain predetermined units, the unit of the medical event isconverted to a different unit, and time-series information about themedical event in the converted unit is newly created. Furthermore, iftime-series information other than binary data, such as real numbers, isincluded in medical information data 51, the time-series information isconverted to binary data in order to make it easy to create attributedata.

As the time-series information about new medical events required forcreation of attribute data, there are three kinds of pieces oftime-series information about “simultaneous medical acts”, “combineddrugs” and “hospital department change”. These new medical eventsrepresent the following meanings, respectively.

The “simultaneous medical acts” is a medical event indicating that alarger number of kinds of medical acts than a predetermined thresholdwere simultaneously performed for a particular patient at a particulartime point. Further, the “combined drugs” is a medical event indicatingthat a larger number of kinds of drugs than a predetermined thresholdwere simultaneously prescribed for a particular patient at a particulartime point. The “hospital department change” is a medical eventindicating that, from a hospital department different from a hospitaldepartment from which a particular drug was prescribed, another drug wasprescribed for a particular patient at a particular time point.

As for the time-series information about the medical events of“simultaneous medical acts”, “combined drugs” and “hospital departmentchange”, it is not necessary to newly create the time-series informationif the time-series information is directly included in inputted medicalinformation data such as itemized statements of medical fee and medicalexamination record information. If the time-series information is notincluded in the medical information data, however, it is necessary toperform preprocessing for the medical information data and newly createthose medical events. A method for creating the time-series informationabout the three medical events of “simultaneous medical acts”, “combineddrugs” and “hospital department change” from the medical informationdata will be described below.

An example of creating time-series information about the medical eventof “simultaneous medical acts” as preprocessing will be shown. It isassumed that a plurality of pieces of time-series information exists forthe same patient for each classification of medical act as shown under(Before conversion). By counting how many medical acts the patientreceived each month, time-series information about the medical event of“simultaneous medical acts” can be created as shown under (Afterconversion). In the example below, description will be made on a casewhere the threshold is set to 20, 10, 7 and 5. The threshold is one ofcontrol parameters 69.

(Before Conversion)

Patient X; Medical act A: 0, 0, 1, 0, 0, 1, 0

Patient X; Medical act B: 0, 0, 1, 0, 0, 1, 0

Patient X; Medical act C: 0, 0, 1, 0, 0, 1, 0

Patient X; Medical act D: 0, 0, 1, 0, 1, 0, 0

Patient X; Medical act E: 0, 0, 1, 0, 0, 0, 0

Patient X; Medical act F: 0, 0, 1, 0, 0, 1, 0

Patient X; Medical act G: 0, 0, 1, 0, 0, 1, 0

(After Conversion)

Patient X; Simultaneous medical acts (20 kinds or more): 0, 0, 0, 0, 0,0, 0

Patient X; Simultaneous medical acts (10 kinds or more): 0, 0, 0, 0, 0,0, 0

Patient X; Simultaneous medical acts (7 kinds or more): 0, 0, 1, 0, 0,0, 0

Patient X; Simultaneous medical acts (5 kinds or more): 0, 0, 1, 0, 0,1, 0

Next, an example of creating time-series information about the medicalevent of “combined drugs” as preprocessing will be shown. It is assumedthat a plurality of pieces of time-series information exists for thesame patient for each classification of drug as shown under (Beforeconversion). By counting how many drugs were prescribed for the patienteach month, time-series information about the medical event of “combineddrugs” can be created as shown under (After conversion). In the examplebelow, description will be made on a case where the threshold is set to20, 10, 7 and 5.

(Before Conversion)

Patient X, Drug A; Internal medicine department: 0, 0, 1, 0, 0, 1, 0

Patient X, Drug B; Surgery department: 0, 0, 1, 0, 0, 1, 0

Patient X, Drug C; Internal medicine department: 0, 0, 1, 0, 0, 1, 0

Patient X, Drug D; Internal medicine department: 0, 0, 1, 0, 1, 0, 0

Patient X, Drug E; Internal medicine department: 0, 0, 1, 0, 0, 0, 0

Patient X, Drug F; Internal medicine department: 0, 0, 1, 0, 0, 1, 0

Patient X, Drug G; Internal medicine department: 0, 0, 1, 0, 0, 1, 0

(After Conversion)

Patient X; Combined drugs (20 kinds or more): 0, 0, 0, 0, 0, 0, 0

Patient X; Combined drugs (10 kinds or more): 0, 0, 0, 0, 0, 0, 0

Patient X; Combined drugs (7 kinds or more): 0, 0, 1, 0, 0, 0, 0

Patient X; Combined drugs (5 kinds or more): 0, 0, 1, 0, 0, 1, 0

Next, an example of creating time-series information about the medicalevent of “hospital department change” as preprocessing will be shown. Itis assumed that a plurality of pieces of time-series information existsfor the same patient for each classification of combination of a drugand a hospital department at which the drug was prescribed, as shownunder (Before conversion). By checking, for each combination of a drugand a hospital department at which the drug was prescribed, whether somedrug was prescribed during the same month from a hospital departmentdifferent from the hospital department at which the drug was prescribed,time-series information about the medical event of “hospital departmentchange” can be created as shown under (After conversion). An example ofcreation of the time-series information about the medical event of“hospital department change” will be shown below.

(Before Conversion)

Patient X, Drug A; Internal medicine department: 0, 0, 1, 0, 0, 1, 0

Patient X, Drug B; Surgery department: 0, 1, 0, 0, 0, 1, 0

Patient X, Drug C; Dermatology department: 0, 0, 0, 1, 1, 1, 0

(After Conversion)

Patient X, Hospital department change (Drug A): 0, 1, 0, 1, 1, 0, 0

Patient X, Hospital department change (Drug B): 0, 0, 1, 1, 1, 0, 0

Patient X, Hospital department change (Drug C): 0, 1, 1, 0, 0, 0, 0

As for the time-series information about each medical event included inmedical information data 51 read out from storage device 12, since it isnot necessary to treat the time-series information about the medicalevent in certain predetermined units, the unit of the medical event isconverted to a different unit, and time-series information about themedical event in the converted unit is newly created.

For example, as for time-series information about a disease, time-seriesinformation about new medical events indicating history of the diseasewhich uses ICD10 codes (codes of ICD (International StatisticalClassification of Diseases and Related Health Problems), 10th Edition)is created with the use of a table of conversion to ICD10 codes.Furthermore, as for time-series information about a drug, time-seriesinformation about a new medical event indicating history of prescriptionof the drug which uses ATC (Anatomical Therapeutic ChemicalClassification System) codes is created with the use of a table ofconversion to the ATC codes. Hereinafter, an example of convertingtime-series information about medical events of “disease” to time-seriesinformation using ICD10 codes as preprocessing will be shown as anexample. It is assumed that “cardiomyopathy” or “secondarycardiomyopathy” was diagnosed for patient X on a particular month asshown under (Before conversion). In this case, since “cardiomyopathy”and “secondary cardiomyopathy” correspond to the same ICD10 code I429,time-series information about a disease called I429 can be created asshown under (After conversion).

(Before Conversion)

Patient X; Cardiomyopathy: 0, 0, 1, 0, 0, 0, 0

Patient X; Secondary cardiomyopathy: 0, 0, 0, 0, 0, 1, 0

(After Conversion)

Patient X; Cardiomyopathy: 0, 0, 1, 0, 0, 0, 0

Patient X; Secondary cardiomyopathy: 0, 0, 0, 0, 0, 1, 0

Patient X; I429: 0, 0, 1, 0, 0, 1, 0

If time-series information other than binary data, such as real numbers,is included in medical information data, the time-series information isconverted to binary data in order to make it easy to create attributedata. For example, there may be a case where time-series informationabout the medical event of “medical expenses” is written in medicalinformation data as real number time-series information. Therefore,description will be made on the time-series information about themedical event of “medical expenses” below as an example of a conversiontarget.

In order to detect a drug adverse event, it is more appropriate toconvert the time-series information to information with roughergranularity, such as information about whether high medical expenses orlow medical expenses, than to handle detailed information such as theamount of “medical expenses”. Further, by converting the time-seriesinformation to the 0/1 binary data format like the other medical eventsincluded in medical information data, time-series information includedin the medical information data can be uniformed in the same format, andtherefore, a merit that it becomes easy to handle the medicalinformation data is obtained.

Therefore, by setting a predetermined threshold (for example, 1000) andindicating whether or not the threshold is reached or exceeded by “0”and “1”, data is converted so that the medical event of “medicalexpenses” expressed by whether or not the threshold is reached orexceeded. This threshold is also one of control parameters 69.

An example of the medical event of “medical expenses” before and afterconversion is shown below. In the example below, description will bemade on a case where the threshold is set to 1000 (four digits) and 100(three digits).

(Before Conversion)

Patient X, Medical Expenses: 300, 550, 90, 140, 2500, 600, 0

(After Conversion)

Patient X; Medical expenses (1000 or more): 0, 0, 0, 0, 1, 0, 0

Patient X; Medical expenses (100 or more): 1, 1, 0, 1, 1, 1, 0

(1-2) Step S13

At step S13, attribute data is created. In the present exemplaryembodiment, attribute data X_(n) is generally represented by a vectorconstituted by a plurality of elements. When it is assumed that, foreach combination of “drug and disease”, a number identifying thecombination is given, and it is assumed that the number of elements ofthe first combination (that is, n=1) is 7,

X₁=(0,0,3,2,1,0,0)

is given. This shows that, for the combination of n=1, the value ofattribute item 1 is 0, the value of attribute item 2 is 0, the value ofattribute item 3 is 3, the value of attribute item 4 is 2, the value ofattribute item 5 is 1, the value of attribute item 6 is 0, and the valueof attribute item 7 is 0. The created attribute data is stored intostorage device 12.

The attribute data is data showing, for each combination,characteristics of occurrence and non-occurrence of medical eventsincluded in the medical information data at a time close to a time pointwhen the drug and the disease of the combination co-occur on the samepatient on medical information data. This is created based on theknowledge that, at a timing when an adverse event occurs, it will happento the patient that higher expenses than before occur for the patient,that hospitalization occurs, that a drug which has been prescribed isstopped, that some drug or medical act for treating the adverse event isnewly added, that the patient sees a doctor at a hospital departmentdifferent from the hospital department where the drug has beenprescribed, or the like.

In order to detect a drug adverse event from medical information datawith high accuracy, it is necessary that some difference exists inattribute data between a case where an adverse event has occurred and acase where an adverse event has not occurred. Therefore, it can bethought that, by expressing the characteristics of occurrence andnon-occurrence of a particular medical event at timing when an adverseevent occurs, as attribute data, some difference occurs in the attributedata between the case where an adverse event has occurred and the casewhere an adverse event has not occurred.

Next, the details of a method for creating the attribute data will bedescribed. The attribute data are roughly classified in the followingsix kinds.

(Pattern Attribute Data)

The first is pattern attribute data. This attribute data is attributedata indicating the rate of occurrence of each of patterns in the wholemedical information data. The patterns are obtained as follows. For acombination of a drug and a disease, if the disease occurs on the samepatient within a first period with a predetermined length after the drugis prescribed, patterns indicating order of occurrence andnon-occurrence of predetermined kinds of medical events are extractedwithin a second period before and after the time point of occurrence ofthe disease as a reference, as shown in FIG. 3. The lengths of the firstand second periods are determined by a period length condition which isone of control parameters 69.

This attribute is created based on the idea that some difference willappear in order of occurrence and non-occurrence of medical events, suchas high medical expenses and hospitalization, during a period before andafter occurrence of a disease included in a combination, between a casewhere an adverse event occurs and a case where an adverse event does notoccur.

It can be thought that such a difference occurs, for example, as in thecase where high medical expenses do not occur during a period before adisease included in a combination indicating an adverse event occurs,while the rate of occurrence of such a pattern that high medicalexpenses occur at every time point in succession is high during a periodafter occurrence of the disease, and, on the other hand, the rate ofoccurrence of the pattern that high medical expenses occur at every timepoint in succession is low during a period after occurrence of thedisease included in the combination which is not an adverse event.

A specific example of pattern attribute data will be described. A periodof a total of seven months with a reference time as the center isassumed for a certain patient, and whether hospitalization occurs or notis indicated by “0” (non-occurrence of hospitalization) and “1”(occurrence of hospitalization) for each month of the period. Then, ifthe patient is hospitalized at least once during the seven months, thereare 127 (=2⁷−1) patterns that are represented by a combination of “0”and “1” in total. Therefore, each of the 127 patterns is regarded as anattribute item, and the rate of occurrence of each pattern can be avalue of the attribute item.

(Rate of Occurrence)

The second kind of attribute data is occurrence rate attribute data. Asshown in FIG. 3, this attribute data is attribute data indicating, for acombination of a drug and a disease, the rate of occurrence ofpredetermined kinds of medical event during a period until occurrence ofthe disease after prescription of the drug in the case where the diseaseoccurs within a first period with a predetermined length after the drugis prescribed for the same patient.

This attribute data is created based on the idea that some differencewill appear in the rate of occurrence of medical events such as highmedical expenses and hospitalization during a period until a diseaseoccurs after a drug included in a combination is prescribed, between thecase where an adverse event occurs and the case where an adverse eventdoes not occur.

For example, during a period from a time point when a drug included in acombination indicating an adverse event is prescribed until a time pointwhen a disease occurs, such a case is conceivable that the patientcomplains about bad condition of his or her body because of prescriptionof the drug, and various medical acts are performed at the same time toexamine the body condition of the patient when he receives a medicalexamination by a doctor, so that an event of the number of simultaneousmedical acts (10 or more) occurs. Therefore, it can be thought that therate of occurrence of the medical event of the number of simultaneousmedical acts (10 or more) is high. On the other hand, during a periodfrom a time point when a drug included in a combination which is not anadverse event is prescribed until a time point when a disease occurs, itdoes not happen that the patient complains about bad condition of his orher body because of prescription of the drug and receives a medicalexamination by a doctor. Therefore, it can be thought that the rate ofoccurrence of the medical event of the number of simultaneous medicalacts (10 or more) is low.

(Transition Probability)

The third kind of attribute data is transition probability attributedata. As shown in FIG. 4, this attribute data is attribute dataindicating, for a combination of a drug and a disease, transitionprobability of occurrence or non-occurrence of predetermined kinds ofmedical events during a predetermined third period before occurrence ofthe disease and a predetermined fourth period after the occurrence, withthe time point of the occurrence as a reference, in the case where thedisease occurs within a first period with a predetermined length afterthe drug is prescribed for the same patient.

This attribute data is created based on the idea that some differencewill appear in order of occurrence and non-occurrence of medical eventssuch as high medical expenses and hospitalization during periods beforeand after occurrence of a disease included in a combination, between thecase where an adverse event occurs and the case where an adverse eventdoes not occur.

For example, it can be thought that the probability of transition of themedical event of high medical expenses from non-occurrence to occurrencebecomes high, as in the case where high medical expenses do not occurduring a period before occurrence of a disease included in a combinationindicating an adverse event, while high medical expenses occur afteroccurrence of the disease because of treatment of the adverse event. Onthe other hand, it can be thought that, during periods before and afteroccurrence of a disease included in a combination which is not anadverse event, high medical expenses occur during neither of periods,and therefore, that the probability of the medical event of high medicalexpenses transitioning from non-occurrence to non-occurrence becomeshigh.

(Difference Between Event Occurrence Rates)

The fourth kind of attribute data is event occurrence rate differenceattribute data. This attribute data is such attribute data as describedbelow. For a combination of a drug and a disease, with the use of afirst occurrence rate of predetermined kinds of medical events within apredetermined third period before a time point of occurrence of thedisease as a reference in the case where the disease occurs during afirst period with a predetermined length after the drug is prescribedfor the same patient, and a second occurrence rate of predeterminedkinds of medical events during a predetermined fourth period after that,a null hypothesis that there is not a difference between the firstoccurrence rate and the second occurrence rate and an alternativehypothesis that there is a difference are made, and difference betweenthe rates of the two groups is examined. The fourth kind of attributedata is attribute data indicating a p value indicating the probabilityof the null hypothesis being rejected when the null hypothesis iscorrect.

This attribute is created based on the idea that some difference appearsin the rate of occurrence of medical events such as high medicalexpenses and hospitalization during periods before and after occurrenceof a disease included in a combination, between the case where anadverse event occurs and the case where an adverse event does not occur.

For example, since it can be thought that, though the probability ofoccurrence of high medical expenses is low during a period beforeoccurrence of a disease included in a combination indicating an adverseevent, the probability of occurrence of high medical expenses is highduring a period after occurrence of the disease because of treatment ofthe adverse event, it can be thought that there is a difference in therate of occurrence before and after occurrence of the disease, and the pvalue becomes small. On the other hand, since it can be thought that,during periods before and after occurrence of a disease included in acombination which is not an adverse event, the probability of occurrenceof high medical expenses is low in both of the periods, it can bethought that there is not a difference in the rate of occurrence beforeand after occurrence of the disease, and the p value becomes large.

(Abnormal Value)

The fifth kind of attribute data is abnormal value attribute data. Thisattribute data is such attribute data as described below. For the drugand disease of each inputted combination, with the use of a firstmedical event pattern set obtained by collecting results of extractingpatterns indicating order of occurrence and non-occurrence ofpredetermined kinds of medical events within a predetermined secondperiod before and after a time point when the drug is prescribed for thesame patient as a reference point, and a second medical event patternset obtained by, in the case where the disease occurs within a firstperiod with a predetermined length after the drug is prescribed for thesame patient, collecting results of extracting patterns indicating orderof occurrence and non-occurrence of predetermined kinds of medicalevents within a predetermined second period before and after the timepoint when the disease occurs as a reference, a pattern of medicalevents is learned with a probability model using the first medical eventpattern set. A numerical value indicating how abnormal each patternincluded in the second medical event pattern set is as a patterngenerated from the learned probability model is caused to be theattribute data.

As for this attribute, it can be thought that some difference willappear in order of occurrence and non-occurrence of medical events, suchas high medical expenses and hospitalization, during periods before andafter occurrence of a disease included in a combination, between thecase where an adverse event occurs and the case where an adverse eventdoes not occur.

Furthermore, it is originally rare that an adverse event occurs becauseof prescription of a drug. It can be thought that, in most cases, anadverse event does not occur even if a drug is prescribed. Therefore, itcan be thought that, if the medical event occurrence pattern at thetiming of a drug being prescribed for a patient is learned by aprobability model irrespective of whether an adverse event occurs ornot, a medical event occurrence pattern at a timing when an adverseevent occurs after the drug is prescribed for the patient shows anoccurrence pattern which extremely rarely occurs.

Therefore, it can be thought that, by learning the medical eventoccurrence pattern at the timing of a drug being prescribed for apatient by a probability model and causing a numerical value indicatinga degree of abnormality as a pattern generated from the learnedprobability model to be attribute data, difference will appear in theattribute data between the case where an adverse event occurs and thecase where an adverse event does not occur.

For example, it can be thought that, when a drug is prescribed and anadverse event occurs, a pattern that high medical expenses occur atevery time point in succession accompanying treatment of the adverseevent. On the other hand, it can be thought that such a pattern thathigh medical expenses occur at every time point in succession does noteasily occur in time-series information about the medical event at thetiming of a drug being prescribed for a patient.

As the probability model, for example, a Markov probability model can beused. In this case, by treating a medical event occurrence pattern as aMarkov process and inputting the Markov process into a learned Markovprobability model, attribute data is calculated. Otherwise, a naiveBayesian model can be used as the probability model.

(Outlier)

The sixth kind of attribute data is outlier attribute data. Thisattribute data is attribute data indicating, when the above-describedpattern attribute data about the drug and disease of each inputtedcombination is used to compare the pattern attribute data of eachcombination with pattern attribute data of other inputted combinations,a deviation degree of magnitude tendency of the value of each attributeitem of the pattern attribute data of each combination.

This attribute is created based on the idea that, because an adverseevent is an event that occurs rarely and many of combinations arecombinations that do not indicate adverse events, and because, in themagnitude tendency of the value of each attribute item of patternattribute data, combinations not indicating adverse events resemble oneanother but do not resemble combinations indicating adverse events, themagnitude tendency of the value of each attribute item of attribute dataof a combination indicating an adverse event, among all thecombinations, significantly deviates in comparison with many othercombinations.

For example, as for a combination indicating an adverse event, it can bethought that, if the drug is prescribed and the disease occurs, the rateof occurrence of the pattern that high medical expenses occur at everytime point in succession is high. As for many other combinations notindicating adverse events, however, the rate of occurrence of thepattern that high medical expenses occur at every time point insuccession is low. Therefore, from the viewpoint of the rate ofoccurrence of the pattern that high medical expenses occur at every timepoint in succession, it can be thought that a combination indicating anadverse event deviates in comparison with the other combinations.

A technique of calculating a degree of deviation from others is used tocreate the attribute data. Techniques related thereto are known asoutlier detection techniques. It is a 1-class support vector machine(one-class SVM) that is known as one of general techniques among thetechniques. The 1-class support vector machine is a technique in which adiscriminant model is learned so that data distributed at a high densityamong inputted data is discriminated as a positive example, and apositive value is outputted while data other than the data isdiscriminated as a negative example, and a negative value is outputted;and, when data distributed at a low density (that is, data deviatingfrom the other inputted data) is inputted with the use of the learneddiscriminant model, a negative value is outputted.

As attribute data other than the six kinds described above, attributedata of various modifications exist. The attribute data of themodifications will be described below.

(Indicator Function)

The attribute data of the modifications include, for example, indicatorfunction attribute data. This attribute data is attribute dataindicating, for the drug and disease of each inputted combination, whichICD10 code the disease belongs to.

This attribute is created base on the idea that a disease belonging tothe same ICD10 code as a disease included in a combination indicating anadverse event is similarly an adverse event. Since ICD10 codes are codesfor classifying diseases, it can be thought that, if diseases have thesame ICD10 codes, the types of the diseases resemble each other.Therefore, it can be thought that, if a certain disease is an adverseevent of a drug, other diseases having the same ICD10 code indicateadverse events of the same drug.

A specific method for creating the indicator function attribute datawill be described. From all kinds of drugs and disease names included inpositive examples, negative examples and combinations which are neitherpositive examples nor negative examples read from storage device 12, alist of kinds of unique drugs and a list of kinds of unique diseasenames are created.

Further, each of the disease names included in the list of kinds ofunique disease names is converted to the ICD10 unit with the use of thetable of conversion to ICD10. Furthermore, a list showing the kinds ofunique ICD10 among them is created. Then, a list showing allcombinations of drug and ICD10 is newly created with the use of the listof kinds of unique drugs and the list of kinds of unique ICD10. Forexample, if the number of kinds of drugs is 10, and the number of kindsof ICD10 is 100, then the list shows 1000 combinations of drug andICD10.

Then, for each of combinations of “drug and disease” which are positiveexamples, negative examples or those that are neither positive nornegative examples, the disease included in the combination is convertedto the ICD10 unit with the use of the table of conversion to ICD10 codesfirst, with each of the combinations written in the list as an attributeitem. If the combination of the drug and ICD10 included in thecombination corresponds to a combination of drug and ICD10 shown by anattribute item, the value of “1” is set as the value of the attributeitem. Otherwise, the value of “0” is set as the value of the attributeitem.

The various attribute data which can be used in the present exemplaryembodiment has been described above. However, as the attribute data,those other than the attribute data given here can be also used.Further, in the present exemplary embodiment, it is recommended tochange the kinds of predetermined medical events and similarly createthe above-described attribute data. As the kinds of medical events, forexample, hospitalization, medical expenses (four or more digits),medical expenses (three or more digits), simultaneous medical acts (20or more kinds), simultaneous medical acts (10 or more kinds),simultaneous medical acts (7 or more kinds), simultaneous medical acts(5 or more kinds), the number of combined drugs (20 or more kinds), thenumber of combined drugs (10 or more kinds), the number of combineddrugs (7 or more kinds), the number of combined drugs (5 or more kinds),disease-related medical events after conversion to the ICD10 unit,disease-related medical events after conversion to the ATC unit, and thelike.

Further, the above-described attribute data may be similarly createdaccording to gender or age of a patient. For example, the rate ofoccurrence of a hospitalization event of a male patient in his twentiescan be attribute data. This corresponds to determining attribute dataindicating the rate of occurrence of a predetermined kind of medicalevent during a period until a disease occurs after prescription of adrug in the case where the disease occurs within a first period with apredetermined length after the drug is prescribed for the same malepatient in his twenties.

Further, at the time of determining attribute data about a combinationof “drug and disease”, the attribute data may be created inconsideration of only the first prescription, with regard toprescription of the drug. Further, there may be a case where diseasesoccur at a plurality of points of time during a first period afterprescription of the drug. In this case, such a limitation may be appliedthat the attribute data is created with only the first disease as areference. Otherwise, attribute data may be created, with each of thesecond and succeeding diseases as a reference point, without limitingthe reference point only to the first prescription. Otherwise, acondition limiting prescription of the drug and a condition limiting adisease to be a reference point may be combined.

(2) Details of Learning Phase S2

(2-1) Step S21

At step S21, positive example combinations 52, negative examplecombinations 53, positive/negative example flags 55 and attribute data56 corresponding to the positive and negative examples are read out fromstorage device 12. It is assumed that N positive and negative examplecombinations are read out, and each combination is referred to as acombination number n (n=1, . . . , N). Further, an adverse event flag isindicated by Y_(n) (n=1, . . . , N). That is, Y_(n) is a flag indicatingwhether combination n is a positive example combination (Y_(n)=1) or anegative example combination (Y_(n)=−1). Further, the read-out attributedata is indicated by X_(n) (n=1, 2, . . . , N). As described above,X_(n) is attribute data corresponding to combination n.

(2-2) Step S22

In the present exemplary embodiment, a value to be determined byinputting data to a discriminant model is adverse event score S(X_(n)).At step S22, the discriminant model for calculating the value isspecified, and parameters thereof are learned. Adverse event scoreS(X_(n)) indicates strength of suspicion of an adverse event againstcombination n. Hereinafter, the operation of learning the discriminantmodel for calculating adverse event score S(X_(n)) will be described.

As the discriminant model, it is recommended to use, for example, alinear support vector machine (hereinafter referred to as a linear SVM)capable of, when attribute data X_(n) corresponding to a positiveexample combination with the positive/negative example flag Y_(n)=1 isgiven, outputting a score indicating strength of possibility of Y_(n)=1for certain X. The linear SVM is a model which is often applied to abinary discrimination problem for discriminating whether Y_(n)=1 orY_(n)=−1 from X. Further, other discriminant models such as a logisticregression model may be used.

The operation of learning the discriminant model will be described withthe linear SVM as an example.

In the linear SVM, the following linear discriminant function is used,which discriminates between a positive example and a negative example byoutputting a positive value to attribute data of a positive examplecombination and outputting a negative value to attribute data of anegative example combination when a weight vector W is an M-dimensionalweight vector.f(X _(n) ,W)=W ^(T) X _(n)  (1)wherein the superscript “T” indicates transposition of the vector.

When (X_(n), Y_(n)), (n=1, . . . , N) related to positive and negativeexample combinations are given as learning data for the discriminantmodel, the value of weight vector W is calculated by minimizing thefollowing objective function in the linear SVM.

$\begin{matrix}{{L(W)} = {{C{\sum\limits_{n = 1}^{N}\left( \left( {\max\left( {0,{1 - {Y_{n}W^{T}X_{n}}}} \right)} \right)^{2} \right)}} + {W}}} & (2)\end{matrix}$

The first term on the right side indicates the sum of discriminationerrors. When the signs of Y_(n) and W^(T)X_(n) correspond to each other,the error is zero. When the signs do not correspond to each other,however, the first term on the right side increases by an amountcorresponding to the error. The second term on the right side indicatesa penalty term, and |W| indicates the norm of W. In general, norm 2 ornorm 1 is used. Parameter C is a parameter which adjusts balance betweenthe first term (by reducing error of discrimination between a positiveexample and a negative example) and the second term (a penalty term).Parameter C may be given in advance as one of control parameters 69.Otherwise, a plurality of candidates for parameter C may be given sothat an optimum C may be automatically selected with the use of across-validation method.

The value of a parameter which minimizes L(W) is indicated by W*, andparameters of the discriminant model are assumed to be W*. As a methodfor determining W which minimizes L(W), various optimization techniquesare proposed. For example, a method described in [NPL2] and the likeexist.

The discriminant model, which is a processing result of learning phaseS2, is expressed (defined) by the above learned model parameters W*.

(3) Details of Adverse Event Score Calculation Phase S3

(3-1) Step S31

At step S31, learned discriminant model 57, combinations other thanpositive and negative examples 54 and attribute data corresponding tothe combinations are read out from storage device 12. It is assumed thatK combinations other than positive and negative examples and K pieces ofattribute data corresponding to the combinations are read out. Here, theread-out attribute data is indicated by X_(k) (k=1, 2, . . . , N).

(3-2) Step S32

At step S32, adverse event score S(X_(k)) of X_(k) is calculated bycalled discriminant model W*. Specifically, adverse event score S(X_(k))is calculated as follows:S(X _(k))=W* ^(T) X _(k)  (3)Calculated adverse event score S(X_(k)) is stored into storage device12.(4) Details of Extraction Phase S4(4-1) Step S41

At step S41, the K combinations other than positive and negativeexamples, adverse event score S(X_(k)) (k=1, K) corresponding to thecombinations and an extraction condition are read out from storagedevice 12. Here, for example, the maximum number of combinations to beextracted or a threshold for the adverse event score is used as theextraction condition.

(4-2) Step S42

If the maximum number H of combinations to be extracted is used as theextraction condition, the combinations are sorted according to theadverse event score, and H combinations are extracted in descendingorder of adverse event scores. Further, if a threshold T for the adverseevent score is used as the condition, the combinations are sorted by theadverse event score, and combinations having an adverse event score witha value equal to or larger than T are extracted in descending order.

(4-3) Step S43

At step S43, a list of combinations strongly suspected to be adverseevents, which is an extraction result of extraction at step S42, isstored into storage device 12. Otherwise, the extraction result isoutputted to screen display unit 15 or to the outside via communicationinterface unit 13.

Thus, according to the present exemplary embodiment, it is possible to,on the basis of medical information data, extract a combination of anadverse event from among combinations other than positive and negativeexamples based on adverse event scores indicating suspicion as anadverse event so that an extraction condition is satisfied.

In the present exemplary embodiment, at the time of determiningattribute data from time-series information about medical events withina predetermined period, targeting combinations of “drug and disease”,those include a drug having been prescribed for a patient and a diseasehaving been observed in the patient as well as at least one of a medicalact performed for the patient and an event showing that the medical acthas been performed, accompanying the medical act, are used as themedical events. If an adverse event occurs on the patient, the patientsees a doctor at any hospital department to treat it, a medical act isperformed to examine/treat it, and the patient is charged for medicalexpenses for the medical act. Therefore, some difference appears intime-series information about the medical events of medical act,hospitalization, medical expenses and hospital department between thecase where an adverse event has occurred and the case where an adverseevent has not occurred.

Therefore, even if, for a combination which is an adverse event and acombination which is not an adverse event, attribute data about thenumber of times that a disease occurs during a drug prescription periodhave the same content, it becomes possible to prevent attribute dataabout the combination which is an adverse event and the combinationwhich is not an adverse event from having the same content by creatingvarious kinds of attribute data about occurrence of the medical eventsof medical act, hospital department, medical expenses andhospitalization/non-hospitalization. As will be apparent from thepresent exemplary embodiment, according to the present invention, bycreating attribute data using at least one more kind of medicalinformation in addition to time-series information about a drug andoccurrence of a disease, it becomes possible to widely extract adverseevents with fewer mistakes in comparison with the case of creatingattribute data using only a drug and occurrence of a disease.

In the drug adverse event extraction apparatus of the present exemplaryembodiment described above, each of input unit 21, attribute datacreation unit 22, discriminant model learning unit 23, adverse eventscore calculation unit 24 and extraction unit 25 provided in processingapparatus 11 can be configured as dedicated hardware. Otherwise, thewhole processing apparatus 11 can be configured with a computer providedwith a microprocessor such as a CPU (central processing unit) and itsperipheral circuits. If processing apparatus 11 is realized by thecomputer, the computer can be caused to read and execute a program forexecuting the functions of input unit 21, attribute data creation unit22, discriminant model learning unit 23, adverse event score calculationunit 24 and extraction unit 25 described above. The program is read froman external apparatus via communication interface unit 13 and the likeor read from a computer-readable storage medium and stored into storagedevice 12 or a memory for the program provided separately from storagedevice 12 in advance. Furthermore, all or a part of creation ofattribute data, learning of a discriminant model, calculation of adverseevent scores and extraction of combinations corresponding to orindicating adverse events may be distributed to and executed by aplurality of processors.

An exemplary embodiment has been described above. The present invention,however, is not limited to the above exemplary embodiment, and variouskinds of additions and changes are possible. Various modifications ofthe exemplary embodiment described above will be described below. In adrug adverse event extraction apparatus according to each modificationalso, processing apparatus 11 can be realized by causing a computer toread and execute a corresponding program.

[Modification 1]

FIG. 5 shows a configuration of a drug adverse event extractionapparatus according to Modification 1. In the drug adverse eventextraction apparatus shown in FIG. 1, it is necessary to input threekinds of positive example combinations, negative example combinationsand combinations other than positive and negative examples in advance.By the way, time-series information about medical events for eachpatient included in medical information data 51 also includestime-series information about drug prescription and time-seriesinformation about which disease occurred when. Therefore, if there issome mechanism for classifying the positive examples, the negativeexamples and the combinations other than positive and negative examples,extraction of drug adverse information events must be performed withoutinputting the three kinds of the positive example combinations, negativeexample combinations and the combinations other than positive andnegative examples in advance. Therefore, what is shown in FIG. 4 is suchthat, in the apparatus shown in FIG. 1, positive/negative exampledictionary 61 in which, for each combination of “drug and disease”,whether the combination is to be classified as a positive example or anegative example is written is arranged in storage device 12, andcombination extraction unit 26 that extracts a combination of “drug anddisease” from time-series information about medical events for eachpatient included in medical information data 51 and automaticallyperforms classification about whether the extracted combination is apositive example, a negative example or a combination other thanpositive and negative examples based on positive/negative exampledictionary 61 is provided in processing apparatus 11. Combinationextraction unit 26 corresponds to combination extracting means.

FIG. 6 shows the operation of the drug adverse event extractionapparatus of Modification 1 shown in FIG. 6. In this apparatus, beforeattribute data creation phase S1 in the operation shown in FIG. 2,combination extraction phase S5 is provided at which combinations of“drug and disease” are extracted from medical information data, and theextracted combinations are classified into positive examplecombinations, negative example combinations and combinations other thanpositive and negative examples with reference to positive/negativeexample dictionary 61.

At combination extraction phase S5, input unit 21 first receives medicalinformation data, target drugs and the positive/negative exampledictionary and stores them into storage device 12 at step S51. Next, atstep S52, combination extraction unit 26 extracts a combination frommedical information data 51. At that time, combination extraction unit26 refers to the period length condition included in control parameters69, and, if, for example, a certain patient has a disease which occurredduring the first period (see FIG. 3) after prescription of a certaindrug, extracts the combination of the drug and the disease as a target.Then, combination extraction unit 26 judges whether the extractedcombination is a positive example or a negative example with referenceto positive/negative example dictionary 61, stores the combination intostorage device 12 as a positive example combination if the combinationis a positive example, stores the combination into storage device 12 asa negative example combination if the combination is a negative example,and, in other cases, stores the combination into storage device 12 as acombination other than positive and negative examples.

After step S52 ends, attribute data creation phase S1, learning phaseS2, adverse event score calculation phase S3 and extraction phase S4 aresequentially executed similarly to the case shown in FIG. 2.

[Modification 2]

FIG. 7 shows a configuration of a drug adverse event extractionapparatus according to Modification 2. In extraction of drug adverseevents, there may be a disease which is thought not to be related to acertain drug at all. When attempting to extract adverse events,targeting combinations of “drug and disease” including such a disease,there is a possibility that operation time is increased or accuracydegradation is caused. Therefore, the drug adverse event extractionapparatus shown in FIG. 7 is such that positive/negative example stopword dictionary 62 is stored in storage device 12 instead of thepositive/negative example dictionary in the apparatus shown in FIG. 5.In what is shown in FIG. 7, positive/negative example stop worddictionary 62 is such that a list of diseases not to be thought as beingincluded in processing target combinations is further stored as a listof stop words in positive/negative example dictionary 61 inModification 1. In Modification 2, by using positive/negative examplestop word dictionary 62, use of a combination which includes a diseasecorresponding to a stop word is prevented.

FIG. 8 shows the operation of the drug adverse event extractionapparatus of Modification 2 shown in FIG. 7. In the operation inModification 2, combination extraction phase S5 a is provided at which,by referring to positive/negative example stop word dictionary 62,unnecessary combinations are removed from among combinations of “drugand disease” extracted from medical information data, and the remainingcombinations are extracted as positive examples or negative examples,instead of combination extraction phase S5 in the operation ofModification 1 shown in FIG. 5. At combination extraction phase S5 a,input unit 21 first receives medical information data, target drugs, aperiod length condition and the positive/negative example stop worddictionary and stores them into storage device 12 at step S53. Next, atstep S54, combination extraction unit 26 extracts a combination frommedical information data 51. At that time, combination extraction unit26 refers to the period length condition included in control parameters69, and, if, for example, a certain patient has a disease which occurredduring the first period (see FIG. 3) after prescription of a certaindrug, extracts the combination of the drug and the disease as a target.Then, combination extraction unit 26 judges whether the extractedcombination is a stop word or not with reference to positive/negativeexample stop word dictionary 62 first. If the combination is a stopword, combination extraction unit 26 excludes the combination, and, foreach of the remaining combinations, judges whether the combination is apositive example or a negative example with reference topositive/negative example stop word dictionary 62. In the case of apositive example, the combination is stored into storage device 12 as apositive example combination. In the case of a negative example, thecombination is stored into storage device 12 as a negative examplecombination. In other cases, the combination is stored into storagedevice 12 as a combination other than positive and negative examples.

After step S54 ends, attribute data creation phase S1, learning phaseS2, adverse event score calculation phase S3 and extraction phase S4 aresequentially executed similarly to the case shown in FIG. 2.

[Modification 3]

FIG. 9 shows a configuration of a drug adverse event extractionapparatus according to Modification 3. In the examples described so far,a single discriminant model is used for a plurality of kinds of drugs.In Modification 3, however, a different discriminant model is used foreach kind of drug. Therefore, a plurality of discriminant models by drug63 and a plurality of adverse event scores by drug 64 are stored instorage device 12 instead of discriminant model 57 and adverse eventscores 58 in the apparatus shown in FIG. 1.

FIG. 10 shows the operation of the drug adverse event extractionapparatus of Modification 3 shown in FIG. 9. In the operation inModification 3, instead of learning phase S2 and adverse event scorecalculation phase S3 in the operation shown in FIG. 2, learning phase S2a and adverse event score calculation phase S3 a are provided,respectively. At learning phase S2 a, after step S21 of learning phaseS2 in FIG. 2 (however, discriminant models by drug 63 are called insteadof discriminant model 57) is performed, discriminant model learning unit23 divides positive example combinations and negative examplecombinations according to the kinds of drugs at step S23. Then, at step24, according to the kinds of drugs, discriminant model learning unit 23learns corresponding discriminant models by drug 63 and stores resultsinto storage device 12. Further, at adverse event score calculationphase S3 a, after step S31 of adverse event score calculation phase S3in FIG. 2 is executed, adverse event score calculation unit 24 appliesattribute data to corresponding discriminant models by drug 63 accordingto the kinds of drugs and calculates adverse event scores by drug. Thecalculated adverse event scores by drug are stored into storage device12.

Further, in Modification 3, a multi-task learning technique of learningdiscriminant models by drug not separately but simultaneously may beadopted at learning phase S2 a. The multi-task learning technique is alearning technique for simultaneously learning a plurality of relatedmodels (in the present exemplary embodiment, discriminant models), andit is known that it may be possible to learn models so that theperformance of each model becomes higher (with regard to the presentexemplary embodiment, the performance of each discriminant modeldiscriminating between positive and negative examples becomes higher)than learning the models separately. A representative example of themulti-task learning technique is described in [NPL3]. There are variouskinds of multi-task learning techniques. Trace-norm regularizedmulti-task learning, which is one of the most common methods among them,may be used. This method is a method for learning model parameters ofeach of a plurality of models so that the model parameters exist in alow-dimensional space common to the plurality of models.

Furthermore, it is known that, in the multi-task learning technique, ifthere is somewhat strong relation among models (in this exemplaryembodiment, discriminant models) to be simultaneously learned, themodels can be learned so that the performance of each of the models (inthe example of the present exemplary embodiment, the performance of eachdiscriminant model discriminating between positive and negativeexamples) becomes higher. Therefore, in the present exemplaryembodiment, it is recommended not to adopt the multi-task learningtechnique for learning of all the discriminant models by drug but toadopt the multi-task learning technique for each group of discriminantmodels by drug related to drugs with the same efficacy. It can bethought that, by doing so, a more advantageous effect of adopting themulti-task technique is obtained. This is because it can be thoughtthat, if drugs have the same efficacy, the kinds of adverse eventscaused by prescription of the drugs resemble one another.

Furthermore, in Modification 3, a positive/negative example dictionarymay be provided so that positive example combinations and negativeexample combinations are automatically extracted, similarly toModification 1. A positive/negative example stop word dictionary similarto that of Modification 2 may be provided so that unnecessarycombinations are not used while positive example combinations andnegative example combinations are automatically extracted.

[Modification 3-1]

In Modification 3, a different discriminant model is used for each kindof drug. As a variation thereof, a different discriminant model is usedfor each efficacy in Modification 3-1. That is, a single discriminantmodel is used for a plurality of drugs having the same efficacy. ThisModification 3-1 corresponds to such that uses not brands of drugs but“efficacies of drugs” as “the kinds of drugs”. Therefore, a plurality ofdiscriminant models by efficacy and a plurality of adverse event scoresby efficacy are stored in storage device 12 instead of the plurality ofdiscriminant models by drug 63 and the plurality of adverse event scoresby drug 64 in the apparatus shown in FIG. 9.

The operation of the drug adverse event extraction apparatus ofModification 3-1 is similar to the operation in Modification 3 shown inFIG. 10. However, learning of discriminant models by efficacy isperformed instead of learning of discriminant models by drug at stepS24, and calculation of adverse event scores by efficacy is performedinstead of calculation of adverse event scores by drug at step S33.

[Modification 3-2]

In the examples described so far, positive and negative examplecombinations are learned with discriminant models irrespective of thefrequency of the positive and negative example combinations on medicalinformation data. In Modification 3-2, however, positive and negativeexample combinations that appear with a high frequency and positive andnegative example combinations that appear with a low frequency arelearned with separate discriminant models. Therefore, discriminantmodels by high/low frequency and adverse event scores by high/lowfrequency are stored in storage device 12 instead of the plurality ofdiscriminant models by drugs 63 and the plurality of adverse eventscores by drug 64 in the apparatus shown in FIG. 9.

In the case of learning discriminant models for discriminating positiveexamples and negative examples having a common characteristic in termsof co-occurrence frequency, the performance of discriminating between apositive example and a negative example is thought to become higher byseparately treating combinations of such a drug and a disease thatco-occur frequently and combinations of such a drug and a disease thatco-occur only rarely as shown in Modification 3-2 in comparison withextracting adverse events from a mixture of combinations of such a drugand a disease that co-occur frequently and combinations of such a drugand a disease that co-occur only rarely. For example, since there arevarious kinds of adverse events from a slight adverse event thatfrequently occurs to a serious adverse event that rarely occurs, it canbe thought that, by learning a discriminant model using positive andnegative example combinations of such a drug and a disease thatfrequently co-occur, such an adverse event that frequently occurssimilarly can be detected. On the other hand, it can be thought that, bylearning a discriminant model using positive and negative examplecombinations of such a drug and a disease that rarely co-occur, such anadverse event that rarely occurs can be detected. Thus, by learningdiscriminant models by high/low frequency, it is expected that theaccuracy of detecting adverse events is increased.

The operation of the drug adverse event extraction apparatus inModification 3-2 is similar to the operation in Modification 3 shown inFIG. 10. However, at the time of dividing combinations at step S23, acertain threshold is set, and the combinations are divided ascombinations the frequency of which is equal to or above the thresholdand combinations the frequency of which is below the threshold. At stepS24, instead of learning discriminant models by drug, correspondingdiscriminant models by frequency are learned based on whether highfrequency or low frequency. In calculation of adverse event scores bydrug at step S33, attribute data is applied to correspondingdiscriminant models by frequency based on high/low frequency ofcombinations to calculate adverse event scores by frequency.

As for the frequency threshold used at the learning phase, a pluralityof candidates are prepared in advance, and an optimum thresholddetermined by the cross-validation method can be used.

[Modification 4]

FIG. 11 shows a configuration of a drug adverse event extractionapparatus according to Modification 4. The apparatus shown in FIG. 11 issuch that, in the apparatus shown in FIG. 1, when combinations otherthan positive and negative examples are extracted and sorted in order ofadverse event scores, grouping is performed for the extractedcombinations other than positive and negative examples. In comparisonwith the apparatus shown in FIG. 1, the apparatus shown in FIG. 11 has aconfiguration in which grouping unit 27 is provided in processingapparatus 11, and grouped combinations 65, grouped adverse event scores66 and grouping condition 67 are stored in storage device 12. Groupingunit 27 corresponds to grouping means.

For example, even if the score of each combination indicating an adverseevent that indicates suspicion as an adverse event is low, the score ofa disease group which includes a combination indicating an adverse eventbecomes relatively high in comparison with other disease groups, bygrouping combinations other than positive and negative examples andsumming up scores for each disease group; and it can be thought thatthere is a possibility that the score of the disease group becomeshigher. Therefore, in Modification 4, after outputting an adverse eventscore for each of combinations other than positive and negativeexamples, the disease names of the combinations are grouped with the useof some reference. Then, the scores of a plurality of combinationbelonging to the same disease group are summed up to calculate a groupedadverse event score. Then, the grouped combination and the groupedadverse event score are outputted as a set.

FIG. 12 shows the operation of the drug adverse event extractionapparatus of Modification 4 shown in FIG. 11. The operation inModification 4 is such that grouping phase S6 is provided betweenadverse event score calculation phase S3 and extraction phase S4 in theoperation shown in FIG. 2. At grouping phase S6, grouping unit 27 firstreads out combinations other than positive and negative examples 54,adverse event scores 58 and grouping condition 71 from storage device 12at step S61. At step S62, grouping unit 27 executes grouping ofcombinations other than positive and negative examples 54, and storesgrouped combinations and grouped adverse event scores corresponding tothe grouped combinations into storage device 12. At extraction phase S4following grouping phase S6, combinations suspected to be adverse eventsare extracted based on grouped combinations 65 and grouped adverse eventscores 66.

Further, a positive/negative example dictionary may be provided so thatpositive example combinations and negative example combinations areautomatically extracted, similarly to Modification 1. Apositive/negative example stop word dictionary similar to that ofModification 2 may be provided so that unnecessary combinations are notused while positive example combinations and negative examplecombinations are automatically extracted.

In Modification 4, various conditions are conceivable as a condition forgrouping disease names of combinations. For example, disease names ofcombinations having the same top four digits (detailed classification)of an ICD10 code (a code of ICD (International StatisticalClassification of Diseases and Related Health Problems), 10th Edition)may be included in the same disease group.

Further, as a method for summing up the scores of a plurality ofcombinations included in grouped combinations to calculate a groupedadverse event score, various methods are conceivable. When the methodfor calculating the grouped adverse event score differs, the tendency ofgrouped combinations that are easily extracted as those having a highscore also differs. Therefore, it is necessary to determine thecalculation method according to the purpose of which grouped combinationhaving which nature is to have a high score. Various grouped adverseevent score calculation methods and the characteristics thereof will besimply described below.

For example, a method may be adopted in which a mean value of the scoresof a plurality of combinations which include diseases belonging to thesame ICD10 code (top four digits) is caused to be the grouped adverseevent score. By causing the mean value to be the grouped adverse eventscore, such a group that the scores of combinations belonging to thegroup are averagely high is extracted as having a high score. Otherwise,in the case where the scores of only a part of combinations areextremely high, the group is extracted as having a high score.

Otherwise, a method may be adopted in which the maximum value among thescores of a plurality of combinations which include diseases belongingto the same ICD10 code (top four digits) is caused to be the groupedadverse event score. By causing the maximum value to be the groupedadverse event score, such a group that includes at least one combinationstrongly suspected to be an adverse event (having a score with a largevalue) among combinations belonging to the same group is easilyextracted as having a high score.

Otherwise, a method may be adopted in which the median among the scoresof a plurality of combinations which include diseases belonging to thesame ICD10 code (top four digits) is caused to be the grouped adverseevent score. By causing the median to be the grouped adverse eventscore, such a group that the scores of a plurality of combinationsbelonging to the same group are high as a whole is extracted as having ahigh score. In the case where the adverse event scores of only a part ofcombinations are extremely high, the group is hardly extracted as havinga high score.

Otherwise, a method may be adopted in which a value obtained by summingup only positive scores among the scores of a plurality of combinationswhich include diseases belonging to the same ICD10 code (top fourdigits) is caused to be the grouped adverse event score. By performingcalculation in this way, such a group that a lot of combinations havinga positive high score are included in the same group is easily extractedas having a high score.

Furthermore, a method may be adopted in which, after normalizing adverseevent scores given to all combinations other than positive examples andnegative examples so that the adverse event scores become values withina range of 0 to 1, the total value of the adverse event scores ofcombinations belonging to the same ICD10 code (top four digits) iscaused to be the grouped adverse event score. By taking the total valueafter normalization, such a disease group that a lot of combinationshaving a high score are included in the same group, and a large numberof combinations is included in the same group is easily extracted ashaving a high score. Since the grouping process is performed after allthe values of ranking scores are normalized to be between 0 and 1, thetotal value is not reduced even if such a combination that the value ofthe score before normalization is small (negatively large) exists in thesame group. There is a tendency that the grouped adverse event score ofsuch a group that there are a lot of combinations belonging to the samegroup becomes higher.

Further, a method may be adopted in which, after normalizing adverseevent scores given to all combinations other than positive examples andnegative examples so that the adverse event scores become values withinthe range of 0 to 1, the mean value of the adverse event scores ofcombinations belonging to the same ICD10 code (top four digits) iscaused to be the grouped adverse event score. By taking the mean valueafter normalization, such a combination that the scores of a pluralityof combinations belonging to the same group are high as a whole isextracted as having a high score. Since the grouping process isperformed after all the values of scores are normalized to be between 0and 1, the total value is not reduced even if such a combination thatthe value of the score before normalization is small (negatively large)exists in the same group. By taking the mean value, the tendency thatthe grouped adverse event score of such a combination that there are alot of combinations belonging to the same group becomes higher issuppressed.

Though grouping is performed based on ICD10 codes, especially based onthe top four digits thereof in the above description, grouping may beperformed with the use of a disease classification system other thanICD10 codes, such as ICD9 codes (codes of ICD, 9th Edition).

[Modification 5]

FIG. 13 shows a configuration of a drug adverse event extractionapparatus according to Modification 5. Combinations of “drug anddisease” obtained from medical information data may include those thatare thought to be noise. Therefore, the apparatus of Modification 5 issuch that, in the apparatus shown in FIG. 1, noise combinations 71 arestored in storage device 12 so that, at the time of extracting acombination suspected to be an adverse events, those corresponding tothe noise combinations are excluded.

Here, the noise combination is such a combination that is hardlyconsidered to be an adverse event. For example, such a combination that,for the drug and disease included in the combination, the number ofpatients in whom the disease appears within three months after the firstprescription of the drug is zero can be regarded as noise. This isbecause a disease which has not occurred at all within three monthsafter the first prescription is thought to be hardly suspected to be anadverse event caused by the drug. Further, for example, such acombination that, for the drug and disease included in the combination,the disease has not occurred in any patients within three months afterprescription of the drug may be regarded as noise. This is because adisease which occurred in a patient before prescription of a drug isthought to be hardly suspected to be an adverse event.

FIG. 14 shows the operation of the drug adverse event extractionapparatus of Modification 5 shown in FIG. 13. The operation inModification 5 is such that extraction phase S4 in the operation shownin FIG. 2 is partially changed and caused to be extraction phase S4 a.At extraction phase S4 a, input unit 21 first receives an extractioncondition and noise combinations from communication interface unit 13 oroperation input unit 14 and stores them into storage device 12 at stepS44. Next, at step S45, extraction unit 25 reads out adverse eventscores 58, extraction condition 59 and noise combinations 71 fromstorage device 12, extracts combinations indicating adverse events fromcombinations other than positive and negative examples so that thecombinations do not corresponding to the noise combinations and theextraction condition is satisfied, and stores a result of the extractioninto storage device 12. After that, extraction unit 25 performs step S43shown in FIG. 2, that is, output of the extraction result.

[Modification 6]

FIG. 15 shows a configuration of a drug adverse event extractionapparatus according to Modification 6. In Modification 5 describedabove, noise combinations are given in advance. In Modification 6,however, only a condition for judging that a combination is noise, thatis, a noise condition is given so that noise combinations can beautomatically extracted from among combinations. The apparatus ofModification 6 as above is such that, in the apparatus of Modification5, noise extraction unit 31 that performs combination of noises isprovided in processing apparatus 11, and noise condition 72 is alsostored in storage device 12. Noise extraction unit 31 corresponds tonoise extracting means.

As the noise condition, for example, such a condition can be used that acombination of “drug and disease” is caused to be noise if the number ofpatients in whom the disease has occurred within three months after thefirst prescription of the drug is zero on medical information data. Thisis because a disease which has not occurred at all within three monthsafter the first prescription is thought to be hardly suspected to be anadverse event caused by the drug. Further, for example, a condition thatsuch a combination that the disease has not occurred in any patientswithin three months after prescription of the drug is caused to be noisemay be adopted. This is because a disease which occurred in a patientbefore prescription of a drug is thought to be hardly suspected to be anadverse event.

FIG. 16 shows the operation of the drug adverse event extractionapparatus of Modification 6 shown in FIG. 15. The operation inModification 6 is such that, in the operation in Modification 5 shown inFIG. 14, noise extraction phase S7 is executed before attribute datacreation phase S1, and extraction phase S4 a is partially changed andcaused to be extraction phase S4 b. At noise extraction phase S7, inputunit 21 first receives medical information data and a noise conditionfrom communication interface unit 13 or operation input unit 14 andstores them into storage device 12 at step S71. Next, at step S72, noiseextraction unit 31 reads the medical information data and the noisecondition from storage device 12, searches the medical information datafor what satisfies the noise condition, finds out a combination to benoise based on a result of the search, and stores them into storagedevice 12. On the other hand, at extraction phase S4 b, step 41 ofextraction phase S4 shown in FIG. 2 is executed, and input unit 21receives an extraction condition from communication interface unit 13 oroperation input unit 14 and stores it into storage device 12. Next, atstep S45, extraction unit 25 reads out adverse event scores 58,extraction conditions 59 and noise combinations 71 from storage device12, extracts combinations indicating adverse events from combinationsother than positive and negative examples so that the combinations donot corresponding to the noise combinations and the extraction conditionis satisfied, and stores a result of the extraction into storage device12. After that, extraction unit 25 performs step S43 shown in FIG. 2,that is, output of the extraction result.

[Modification 7]

FIG. 17 shows a configuration of a drug adverse event extractionapparatus according to Modification 7. In the case of using positiveexample combinations and negative example combinations to extract drugadverse events, there may be a case where any one of the number ofpositive examples and the number of negative examples is larger than theother. When there is imbalance between the number of positive examplesand the number of negative examples, there is a possibility that theaccuracy of learning of a discriminant model is degraded, and theaccuracy of extraction of drug adverse events is degraded. Therefore,Modification 7 is such that, in the drug adverse event extractionapparatus shown in FIG. 1, imbalance correction unit 32 that correctsimbalance in the number between positive examples and negative examplesis provided in processing apparatus 11, and corrected positive examplecombinations 73 and corrected negative example combinations 74 arestored in storage device 12, in order to correct the imbalance betweenthe number of positive example combinations and the number of negativeexample combinations. Imbalance correction unit 32 corresponds tocorrection means, and it corrects imbalance in the number betweenpositive examples and negative examples by generating pseudo examples(pseudo positive examples or pseudo negative examples) for positiveexamples or negative examples the number of which is smaller than theother, or deleting some combinations from positive examples or negativeexamples the number of which is larger than the other. The imbalance inthe number between positive examples and negative examples may becorrected by generating both of pseudo positive examples and pseudonegative examples in a manner the number of generated examples differs,or the imbalance in the number between positive examples and negativeexamples may be corrected by deleting both of positive examples andnegative examples in a manner the number of deleted examples differs.Positive example combinations and negative example combinations aftersuch correction is performed are referred to as corrected positiveexample combinations and corrected negative example combinations,respectively. At the time of learning discriminant model 57, learning ofdiscriminant model 57 is to be performed so that the number is notimbalanced between positive examples and negative examples by usingcorrected positive example combinations 73 and corrected negativeexample combinations 74.

FIG. 18 shows the operation of the drug adverse event extractionapparatus of Modification 7 shown in FIG. 17. The operation inModification 7 is such that, in the operation shown in FIG. 2, imbalancecorrection phase S8 is executed immediately after attribute datacreation phase S1, and learning phase S2 b at which learning of adiscriminant model is performed with the use of corrected positiveexamples and corrected negative examples is provided instead of learningphase S2.

At imbalance correction phase S8, imbalance correction unit 32 firstreceives positive example combinations 52, negative example combinations53, attribute data 56 thereof and positive/negative example flags 55from storage device 12 at step S81. Next, at step S82, imbalancecorrection unit 32 executes an imbalance correction process forgenerating at least either pseudo positive examples or pseudo negativeexamples and storing the pseudo examples into storage device 12. Thepseudo positive example combinations and pseudo negative examplecombinations are not derived from actual medical information data butare data generated as pseudo combinations of “drug and disease” so thatthey can be used for learning of a discriminant model as positiveexample combinations and negative example combinations. As a method forgenerating pseudo positive examples and negative examples, for example,the method described in [NPL4] can be used.

Otherwise, at step S82, imbalance correction unit 32 executes animbalance correction process for performing correction of deletingcombinations from the positive examples or negative examples the numberof which is larger than the other and storing combinations into storagedevice 12. As a method about how to select combinations to be deletedfrom the positive examples or negative examples the number of which islarger than the other, for example, the method described in [NPL5] canbe used.

When generation of pseudo examples or deletion of combinations isperformed for either positive examples or negative examples at step S82,the positive examples or the negative examples for which such generationof pseudo examples or deletion of combinations are not performed arecaused to be corrected combinations as they are in the original state.

At learning phase S2 b, discriminant model learning unit 23 callscorrected positive example combinations 73, corrected negative examplecombinations 74, attribute data 56 corresponding to these combinations,positive/negative example flags 55 and discriminant model 57 fromstorage device 12 at step S25, and, with the use of these, learns thediscriminant model at step S22 similarly to the case shown in FIG. 2.The learned discriminant model is returned to storage device 12.Following learning step S2 b, adverse event score calculation phase S3and extraction phase S4 shown in FIG. 2 are executed as they are.

[Modification 8]

FIG. 19 shows a configuration of a drug adverse event extractionapparatus according to Modification 8. The examples described so far aresuch that make it possible to extract a drug adverse event with a highaccuracy by learning a discriminant model using positive examplecombinations and negative example combinations, and giving a highadverse event score to a combination the attribute data of whichresembles that of a positive example, among combinations other thanpositive and negative examples, on the assumption that distribution oflearning data (positive example combinations and negative examplecombinations) on an attribute data space and distribution of evaluationdata (combinations other than positive and negative examples) resembleeach other. However, if there is a difference between the distributionof learning data on the attribute data space and the distribution ofevaluation data, there is a possibility that, even if the discriminantmodel is learned with the use of the positive and negative examplecombinations as they are, it becomes difficult to extract a drug adverseevent from among the combinations other than positive and negativeexamples with a high accuracy using the discriminant model.

On the other hand, though positive example combinations already known asadverse events and negative example combinations already known as notbeing adverse events are used as learning data, and, for eachcombination, attribute data is created from medical information data inthe present invention, it can be thought that this medical informationdata is influenced by history of medical events performed by a doctor tosuppress appearance of an already-known adverse event, history of amedical event performed in a situation that a doctor knows that acombination is not an adverse event, and the like. Therefore, it can bethought that there is some difference in the distribution on theattribute data space between positive example combinations or negativeexample combinations and such combinations that whether they are adverseevents or not is not clearly known yet.

The situation that there is a difference between the distribution oflearning data on the attribute data space and the distribution ofevaluation data can be interpreted as a situation that a covariate shifthas occurred in which, though the regularity of an output (correspondingto an adverse event score in the present exemplary embodiment) to agiven input (corresponding to a combination of “drug and disease” in thepresent exemplary embodiment) does not differ between learning data(positive and negative example combinations used to learn a discriminantmodel) and evaluation data (combinations other than positive andnegative examples), distribution of given learning data and distributionof evaluation data on attribute data space differ.

Therefore, in Modification 8, a covariate shift learning method isapplied which is known as a learning technique capable of discriminatingbetween a positive example and a negative example using a discriminantmodel with a high accuracy even when a covariate shift has occurred.

In order to apply the covariate shift learning technique, it isnecessary to know distribution of evaluation data in advance at the timeof learning a discriminant model using learning data. In the techniqueof the present exemplary embodiment, however, evaluation data(combinations other than positive and negative examples) is known inadvance, and, therefore, the covariate shift learning technique can beapplied. In general machine learning problems, the situation thatevaluation data is known in advance is rare, and it is one of thefeatures of the present exemplary embodiment that evaluation data isknown in advance.

A typical example of the covariate shift learning technique is describedin [NPL6]. This technique is a technique in which learning data locatedin a high density area of evaluation data on the attribute data space isweighted much, and a discriminant model is learned in consideration ofthe weight. That is, the technique is a method in which highly weightedlearning data for which evaluation data and attribute data resemble eachother is learned being focused on, and slightly weighted learning datafor which attribute data does not resemble evaluation data is preventedfrom being reflected much on learning of a discriminant model.

If the density of combination x in evaluation data is indicated byp_(test)(x), and the density of combination x in learning data isindicated by p_(train)(x), then weight W(x) of combination x is derivedfrom the expression below.

${W(x)} = \frac{p_{test}(x)}{p_{train}(x)}$

The above expression shows that higher weight is given to combination xwhen the density of the evaluation data on the attribute data space ishigh. In other words, the higher the density of the evaluation data onthe attribute data space is, the higher weight combination x generallycosts much, a method for suppressing the calculation cost is needed. Atechnique of suppressing the calculation cost by directly estimatingdensity W(x) without directly estimating p_(test)(x) and p_(train)(x) isdescribed in [NPL6]. In Modification 8, a technique like that of [NPL6]may be used as a learning technique under covariate shift.

Further, in Modification 8, a method may be used in which positive andnegative example combinations that are homogeneous to combinations otherthan positive and negative examples are searched for, and these positiveand negative example combinations are learned being focused on, withoutestimating densities or a density ratio. In this method, the homogeneouspositive and negative example combinations are searched for, based on aEuclidean distance in an attribute data space. Hereinafter, theapparatus and its operation in Modification 8 will be described, withthe case of using this method as an example.

The apparatus of Modification 8 is such that corrected positive examplecombinations and corrected negative example combinations are generatedby a covariate shift process and such that covariate shift processingunit 33 that executes the covariate shift process is provided instead ofimbalance correction unit 32 in the apparatus shown in FIG. 17. Thecorrected positive example combinations and the corrected negativeexample combinations in Modification 8 are obtained by generating andadding pseudo positive or pseudo negative example combinations or bydeleting a part of original positive and negative example combinationssimilarly to the case of Modification 7. Covariate shift unit 33corresponds to covariate shift means. Since attribute data is indicatedby a multidimensional vector as described above, a vector space whichincludes this multidimensional vector is called an attribute data space.Then, in the case of creating pseudo positive and pseudo negativeexample combinations by the covariate shift process, it is favorable tocreate each combination so that attribute data of the pseudo positiveexamples and pseudo negative examples is located in the vicinity of theexistence of attribute data of combinations other than positive andnegative examples in an attribute data space. In the case of deleting apart of original positive and negative example combinations by thecovariate shift process, it is preferable to delete positive andnegative example combinations corresponding to attribute data far awayfrom the attribute data of the combinations other than positive andnegative examples in the attribute data space.

FIG. 20 shows the operation of the drug adverse event extractionapparatus of Modification 8 shown in FIG. 19. The operation ofModification 8 is such that imbalance correction phase S8 in theoperation of Modification 7 shown in FIG. 18 is replaced with covariateshift phase S9. At covariate shift phase S9, covariate shift processingunit 33 first receives positive example combinations 52, negativeexample combinations 53, attribute data 56 thereof, andpositive/negative example flags 55 from storage device 12 at step S91.Next, at step S92, covariate shift processing unit 33 generates at leasteither pseudo positive example combinations or pseudo negative examplecombinations by the covariate shift process, and stores them intostorage device 12 together with original positive and negative examplecombinations as corrected positive example combinations 73 and correctednegative example combinations 74. Otherwise, at step S92, covariateshift unit 33 performs correction of deleting a part of the positive andnegative example combinations, and stores positive and negative examplecombinations after the partial deletion is performed, into storagedevice 12 as corrected positive example combinations 73 and correctednegative example combinations 74. It is also possible to performgeneration/addition of the pseudo positive and pseudo negative examplecombinations and deletion of a part of the original positive andnegative example combinations at the same time, and causes the result tobe corrected positive example combinations 73 and corrected negativeexample combinations 74.

Here, the flow of the covariate shift process will be simply described.

Attribute data of combination x is assumed to be a d-dimensional vector,and the k-th vector element of x is indicated by x_(k). Evaluation datashowing combinations other than positive and negative examples isindicated by D_(test), and learning data indicating positive andnegative example combinations is indicated by D_(train).

Centroid x^(c) of evaluation data D_(test) is a vector for which eachvector element is calculated by:

$x_{k}^{c} = {\frac{1}{n}{\sum\limits_{i = 1}^{n}\; x_{k}^{i}}}$Here, it is assumed that n indicates the number of pieces of evaluationdata (that is, the number of combinations other than positive andnegative examples). Mean Centroid distance “mean” of evaluation dataD_(test) is the average of the distance from the evaluation data to thecentroid calculated by the following expression:

${mean} = {\frac{1}{n}{\sum\limits_{i = 1}^{n}{{dist}\left( {x^{i},x^{c}} \right)}}}$Here, the Euclidean distance of sample x^(i) and sample x^(j) isindicated by dist(x^(i),x^(j)), and, specifically, calculated by thefollowing expression:dist(x ^(i) ,x ^(j))=√{square root over (Σ_(k=1) ^(d)(x _(k) ^(i) −x_(k) ^(j)))}Centroid distance deviation “stdev” of test data D_(test) is thestandard deviation of the distance from the evaluation data to thecentroid calculated by the following expression:

${stdev} = \sqrt{\frac{1}{n}{\sum\limits_{i = 1}^{n}\left( {{{dist}\left( {x^{i},x^{c}} \right)} - {mean}} \right)^{2}}}$

Similarly, it is assumed that median centroid distance “median” ofevaluation data D_(test) is the median of distance from the evaluationdata to the centroid. In calculation of centroid distance deviation“stdev”, “mean” may be used, or “median” may be used instead of “mean”.

In this method, the Euclidean distance space is used, and, by regardingsuch learning data that the Euclidean distance to the centroid ofevaluation data is short as learning data which is also close todistribution of the evaluation data, this learning data is learned beingfocused on. Moreover, for further simplification, it is assumed that thenumber of the values of weight is only three, that is, the values ofweight are 2, 1 and 0. Weight 2 is given to the learning data to belearned being focused on, and weight 0 is given to learning data whichis, on the contrary, not desired to be reflected on a discriminantmodel. Weight 1 is given to other learning data. As for the learningdata with the weight 2, duplicates (that is, pseudo positive or pseudonegative example combinations) are created and added to the learningdata, and the learning data with the weight 0 is removed (that is,corresponding positive or negative example combinations are deleted).The calculation procedure is as follows.

1: Centroid x^(c), mean centroid distance “mean” and centroid distancedeviation “stdev” of evaluation data are calculated, and weight w(x) oflearning data x is calculated. Specifically, for each learning data x,distance dist(x, x^(c)) to centroid x^(c) is calculated, and weight w(x)is obtained by the following expression.

${w(x)} = \left\{ \begin{matrix}2 & {{{dist}\left( {x,x^{c}} \right)} \leq {{mean} - {a \cdot {stdev}}}} \\1 & {{{mean} - {a \cdot {stdev}}} < {{dist}\left( {x,x^{c}} \right)} \leq {{mean} + {b \cdot {stdev}}}} \\0 & {{{mean} + {b \cdot {stdev}}} < {{dist}\left( {x,x^{c}} \right)}}\end{matrix} \right.$

Here, a and b are positive number parameters. In the above expression,median centroid distance “median” may be used instead of “mean”. Whetherto use “mean” or “median” can be also selected as a parameter. Theseparameters are stored in storage device 12 as control parameters 69.

2: When w(x) is 2, one more x is added to the learning data. When w(x)is 0, x is deleted from the learning data.

As described above, as the covariate shift process, pseudo positiveexample combinations and pseudo negative example combinations are added,or positive example combinations and negative example combinationscorresponding to attribute data far away from attribute data ofcombinations other than positive and negative examples in an attributedata space are deleted.

[Modification 9]

FIG. 21 shows a configuration of a drug adverse event extractionapparatus according to Modification 9. In extraction of a drug adverseevent, it is important to consider whether the adverse event is suchthat brings about serious health damage or not. It is preferable todetect an adverse event which brings about serious health damageearlier. Therefore, in the apparatus of Modification 9 shown in FIG. 21,positive example combinations are classified into combinationscorresponding to serious adverse events (referred to as “positiveexample (serious) combinations) and combinations corresponding tonon-serious adverse events (referred to as “positive example(non-serious) combinations). Whether “serious” or “non-serious” reflectsdifference in the degree of seriousness. Because there are negativeexample combinations also, Modification 3 relates to a ternarydiscrimination problem. Therefore, in the apparatus of Modification 9shown in FIG. 21, positive example (serious) combinations 81, positiveexample (non-serious) combinations 82, positive example (serious)discriminant model 83, positive example (non-serious) discriminant model84 and negative example discriminant model 85 are stored in storagedevice 12 instead of storing positive example combinations 52 anddiscriminant model 57, and, furthermore, positive example (serious)scores 86, positive example (non-serious) scores 87 and negative examplescores 88 are also stored, in comparison with the apparatus shown inFIG. 1.

FIG. 22 shows the operation of the drug adverse event extractionapparatus of Modification 9 shown in FIG. 21. In the operation ofModification 9, instead of learning phase S2 and adverse event scorecalculation phase S3 in the operation shown in FIG. 2, learning phase S2c and adverse event score calculation phase S3 b are provided,respectively.

At learning phase S2 c, discriminant model learning unit 23 callspositive example (serious) combinations 81, positive example(non-serious) combinations 82, negative example combinations 53,attribute data 56 corresponding to the positive examples (serious), thepositive examples (non-serious) and the negative examples,positive/negative example flags 55, positive example (serious)discriminant model 83, positive example (non-serious) discriminant model84 and negative example discriminant model 85 from storage device 12 atstep S26. At step S27, discriminant model learning unit 23 learnspositive example (serious) discriminant model 85, with the positiveexample (serious) combinations as positive examples, and othercombinations (the positive examples (non-serious) and the negativeexamples) as negative examples; learns positive example (non-serious)discriminant model 86, with the positive example (non-serious)combinations as positive examples, and other combinations (the positiveexamples (serious) and the negative examples) as negative examples; andlearns negative example discriminant model 85 with the negative examplecombinations as positive examples, and other combinations (the positiveexamples (serious) and the positive example (non-serious)) as negativeexamples. Learned discriminant models 84 to 86 are returned to storagedevice 12.

At adverse event score calculation phase S3 b, adverse event scorecalculation unit 24 reads out positive example (serious) discriminantmodel 83, positive example (non-serious) discriminant model 84, negativeexample discriminant model 85, combinations other than positive andnegative examples 54, and attribute data corresponding to thecombinations from storage device 12 at step S31.

Next, adverse event score calculation unit 24 applies the read-outattribute data to positive example (serious) discriminant model 83 tocalculate positive example (serious) scores and stores the scores intostorage device 12 at step S34; applies the read-out attribute data topositive example (non-serious) discriminant model 84 to calculatepositive example (non-serious) scores and stores the scores into storagedevice 12 at step S35; and applies the read-out attribute data tonegative example discriminant model 85 to calculate negative examplescores and stores the scores into storage device 12 at step S36.

Lastly, at step S4, extraction unit 25 reads out the combinations otherthan positive and negative examples and the positive example (serious)scores, positive example (non-serious) scores and negative examplescores of the combinations from storage device 12 and sequentiallyextracts the combinations in rule order as shown in the example below.

1: extract such combinations that the positive example (serious) scoreand the positive example (non-serious) score are positive and thenegative example score is negative, in descending order of the positiveexample (serious) scores;

2: extract such combinations that the positive example (serious) scoreis positive and the positive example (non-serious) score and thenegative example score are negative, in descending order of the positiveexample (serious) scores;

3: extract such combinations that the positive example (non-serious)score is positive and the positive example (serious) score and thenegative example score are negative, in descending order of the positiveexample (non-serious) scores;

4: extract such combinations that all of the positive example (serious)score, the positive example (non-serious) score and the negative examplescore are negative, in descending order of the absolute values of thenegative example scores;

5: extract such combinations that the negative example score ispositive, the positive example (serious) score is positive, and thepositive example (non-serious) score is negative, in descending order ofthe positive example (serious) scores;

6: extract such combinations that the negative example score ispositive, the positive example (serious) score is negative, and thepositive example (non-serious) score is positive, in descending order ofthe positive example (non-serious) scores; and

7: extract such combinations that the negative example score ispositive, and the positive example (serious) score and the positiveexample (non-serious) score are negative, in ascending order of thenegative example scores.

The combinations may be sequentially extracted in rule order other thanthe rule order of the method in the above example.

In Modification 9, some seriousness degree is introduced, and positiveexamples are classified in two of “serious” and “non-serious”. Positiveexamples, however, can be classified in three or more levels dependingon the degree of seriousness.

[Modification 10]

FIG. 23 shows a configuration of a drug adverse event extractionapparatus according to Modification 10. In Modification 4, grouping ofextracted combinations is performed. In the apparatus of Modification10, however, grouping is performed at a stage of medical informationdata to be inputted (the grouping is also referred to as priorgrouping). For example, when there are a plurality of drugs showing thesame efficacy, there may be a case where it is preferable to treat theplurality of drugs as one kind of drug. Therefore, the apparatus ofModification 10 is such that, in the apparatus shown in FIG. 1, groupingunit 28 which performs grouping is provided in processing apparatus 11,and grouping condition 67 and grouped medical information data 68 arestored in storage device 12. Grouping unit 28 corresponds to priorgrouping means.

FIG. 24 shows the operation of the drug adverse event extractionapparatus of Modification 10 shown in FIG. 23. The operation ofModification 10 is such that attribute data creation phase S1 acorresponding to grouped medical information data is performed insteadof attribute data creation phase 51 in the operation shown in FIG. 2,and, furthermore, grouping phase S6 a is provided at the preceding stageof attribute data creation phase S1 a for creation of the groupedmedical information data. At grouping phase S6 a, input unit 21 receivesmedical information data, positive examples, negative examples,combinations other than positive and negative examples, and a groupingcondition from communication interface unit 13 or operation input unit14 and stores them into storage device 12 at step S63. At this time, asthe positive examples, the negative examples and the combinations otherthan positive and negative examples, combinations for which grouping canbe performed are used. Next, at step S64, grouping unit 28 reads outmedical information data 51 and grouping condition 67 from storagedevice 12, performs grouping of the medical information data based onthe grouping condition, and stores a result into storage device 12 asgrouped medical information data. At attribution creation phase S1 a,attribute data creation unit 22 first reads out grouped medicalinformation data 68, positive example combinations 42, negative examplecombinations 53, combinations other than positive and negative examples54 and control parameters 69 from storage device 12 at step S14. Next,at step S12, attribute data creation unit 22 performs preprocessing,and, at step S13, creates attribute data corresponding to each of theread-out combinations and stores the created attribute data into storagedevice 12. After that, learning phase S2, adverse event scorecalculation phase S3 and extraction phase in the operation shown in FIG.2 are executed.

In Modification 10, in the case of grouping drugs having the sameefficacy, a list of same efficacy groups (a list of drugs showing thesame efficacy) is used as the grouping condition. Grouping unit 28performs a process for replacing drug names which are included in themedical information data and written in the list of same efficacy groupswith same efficacy group names. At the learning phase, a discriminantmodel is learned, with the positive examples, negative examples andcombinations other than positive and negative examples for which drugnames are indicated by same efficacy group names, in addition to thegrouped medical information data replaced with efficacy group names,used as input. After that, at adverse event score calculation phase S3,adverse event scores are calculated with the use of the learneddiscriminant model.

Further, in Modification 10, grouping of disease names can be performed.In the case of performing grouping of disease names, it is preferable touse ICD10 codes. ICD9 codes and the like can also be used in addition toICD 10 codes. In the case of performing grouping using ICD10 codes, atable of correspondence between ICD10 codes and disease names is used asthe grouping condition, and a process for converting disease namesincluded in the medical information data to ICD 10 codes is executed bygrouping unit 28. At the learning phase, a discriminant model islearned, with the positive examples, negative examples and combinationsother than positive and negative examples for which diseases areindicated by ICD10 codes, in addition to the grouped medical informationdata for which disease names are replaced with ICD10 codes, used asinput. After that, at adverse event score calculation phase S3, adverseevent scores are calculated with the use of the learned discriminantmodel.

Various Modifications of the present exemplary embodiment have beendescribed above. These Modifications can be arbitrarily combined exceptfor those that cannot be combined from a viewpoint of the principle likeModifications 2 and 3.

In the exemplary embodiment and modifications described above, at thetime of determining attribute data for each combination of “drug anddisease” from time-series information about medical events, such medicalevents are used that include not only prescription of a drug for apatient and a disease observed in the patient but also at least one of amedical act performed for the patient and an event indicating that themedical act has been performed, accompanying the medical act. If anadverse event occurs on the patient, the patient sees a doctor at anyhospital department to treat it, a medical act is performed toexamine/treat it, and the patient is charged for medical expenses forthe medical act. Therefore, it is expected that some difference appearsin time-series information about the medical events of medical act,hospitalization, medical expenses, hospital department and the likebetween the case an adverse event has occurred and the case where anadverse event has not occurred. In the exemplary embodiment and eachModification that have been described above, attribute data focusingonly on occurrence of a disease during a prescription period of a drugis not used but attribute data on which such difference depending onoccurrence/non-occurrence of an adverse event is to be reflected isused. Therefore, it becomes possible to extract adverse events morewidely and with fewer mistakes.

The present invention has been described above with reference toexemplary embodiments and modifications thereof. The present invention,however, is not limited to the exemplary embodiments and modificationsdescribed above. Various changes that those skilled in the art canunderstand within the scope of the present invention can be made in theconfiguration and details of the present invention.

The present application claims priority based on JP 2014-57635 filed toJapan on Mar. 20, 2014, all the disclosure of which is incorporatedherein.

A part or all of the exemplary embodiment described above can be writtenas in supplements below but are not limited to the supplementary notesbelow.

[Supplementary Note 1]

A drug adverse event extraction method of extracting a combination of adrug and a disease corresponding to a drug adverse event, the methodcomprising, on assumption that combinations already known ascombinations indicating drug adverse events are regarded as positiveexample combinations, combinations already known as combinations notbeing drug adverse events are regarded as negative example combinations,and given combinations being neither positive example combinations nornegative example combinations are regarded as combinations other thanpositive and negative examples:

generating, using medical information data that includes time-seriesinformation about medical events for each patient, attribute data foreach of the positive example combinations, for each of the negativeexample combinations and for each of the combinations other thanpositive and negative examples, based on the time-series informationabout the medical events;

learning a discriminant model by the attribute data corresponding to thepositive example combinations and the attribute data corresponding tothe negative example combinations;

inputting the attribute data corresponding to the combinations otherthan positive and negative examples to the discriminant model tocalculate scores; and

applying an extraction condition to the score calculated for each of thecombinations other than positive and negative examples to extractcombinations other than positive and negative examples being suspectedto be drug adverse events,

wherein the medical events for each patient include prescription of adrug for the patient and a disease observed in the patient, and

wherein the medical events for each patient further include at least oneof a medical act performed for the patient and an event showing that themedical act has been performed accompanying the medical act performedfor the patient.

[Supplementary Note 2]

The method according to Supplementary Note 1, wherein the medical eventsinclude, for the patient, at least one of a drug newly prescribed, adisease other than diseases specified by corresponding combinations,hospitalization/non-hospitalization, request for medical expenses, and ahospital department the patient has seen a doctor.

[Supplementary Note 3]

The method according to Supplementary Note 1 or 2, wherein the attributedata is data showing, for each of the combinations, characteristics ofoccurrence and non-occurrence of a medical event that is at least one ofa medical act performed for the patient and an event showing that themedical act has been performed, accompanying the medical act performedfor the patient, at a time close to a time point when a drug and diseaseof the combination co-occur on the same patient on the medicalinformation data.

[Supplementary Note 4]

The method according to Supplementary Note 3, wherein, as the attributedata, at least one of the following is used:

a pattern indicating, with a combination of a drug and a diseaseregarded as a combination targeted by attribute data generation in acase where the disease occurs within a predetermined first period afterprescription of the drug, occurrence/non-occurrence of a predeterminedkind of medical event during a second period with a predetermined lengththat includes a time point of the occurrence of the disease, in a timeseries;

an occurrence rate of a predetermined kind of medical event within aperiod from prescription of the drug until occurrence of a disease, witha combination of the drug and the disease regarded as a combinationtargeted by attribute data generation in the case where the diseaseoccurs within the first period;

a transition probability about, with a combination of the drug and adisease regarded as a combination targeted by attribute data generationin the case where the disease occurs within the first period, whether apredetermined kind of medical event occurs during a second period with apredetermined length that includes a time point of the occurrence of thedisease;

probability of, with a combination of the drug and a disease regarded asa combination targeted by attribute data generation in the case wherethe disease occurs within the first period, one of hypotheses beingrejected when the hypothesis is correct, as a result of examination ofthe hypotheses, the hypotheses being a null hypothesis that there is nota difference in the rate of occurrence of a predetermined kind ofmedical event before and after a time point of the occurrence of thedisease and an alternative hypothesis that there is a difference;

a numerical value indicating, for a drug and disease of each inputtedcombination, how abnormal each pattern included in a second medicalevent pattern set is as a pattern generated from a learned probabilitymodel, the numerical value being obtained by learning with theprobability model with the use of: a first medical event pattern set inwhich results of extracting patterns indicating order of occurrence andnon-occurrence of a predetermined kind of medical event within apredetermined third period before and after a time point of prescriptionof the drug for the same patient as a reference point are collected, andthe second medical event pattern set in which results of extractingpatterns indicating order of occurrence and non-occurrence of thepredetermined kind of medical event within the third period before andafter a time point of occurrence of the disease as a reference point inthe case where the disease occurs within the first period after theprescription of the drug for the same patient are collected; and

a degree of, for the pattern attribute data about the drug and diseaseof each inputted combination, how much the pattern attribute data isdeviated in comparison with pattern attribute data of other inputtedcombinations, based on magnitude tendency of the value of each attributeitem in the pattern attribute data of each combination.

[Supplementary Note 5]

The method according to Supplementary Note 3 or 4, wherein the attributedata includes, for the drug and disease of each inputted combination,data indicating which ICD10 code the disease belongs to.

[Supplementary Note 6]

The method according to any one of Supplementary Notes 1 to 5, furthercomprising executing preprocessing for generating time-seriesinformation about a new medical event that is not directly included inthe medical information data but is used for generation of the attributedata, from time-series information about the plurality of medical eventsincluded in the medical information data.

[Supplementary Note 7]

The method according to any one of Supplementary Notes 1 to 6, furthercomprising extracting the combinations from the medical information dataand classifying the extracted combinations into the positive examplecombinations and the negative example combinations based on adictionary, before generating the attribute data.

[Supplementary Note 8]

The method according to Supplementary Note 7, wherein the dictionary inwhich diseases to be ignored are further described is used; and theextracted combinations that include a disease other than the diseases tobe ignored are classified into the positive example combinations and thenegative example combinations.

[Supplementary Note 9]

The method according to any one of Supplementary Notes 1 to 8, whereinthe discriminant model for each kind of drug is used to calculate theadverse event score for the kind of drug.

[Supplementary Note 10]

The method according to any one of Supplementary Notes 1 to 8, wherein adifferent discriminant model is used according to whether frequency ofeach combination in the medical information data is high or low tocalculate the adverse event score according to whether the frequency ishigh or low.

[Supplementary Note 11]

The method according to any one of Supplementary Notes 1 to 10, whereinan extraction result is outputted from combinations strongly suspectedto be drug adverse events, based on the scores.

[Supplementary Note 12]

The method according to Supplementary Note 11, wherein the extractionresult is outputted according to kinds of drugs.

[Supplementary Note 13]

The method according to any one of Supplementary Notes 1 to 10,comprising grouping the extracted combinations other than positive andnegative examples.

[Supplementary Note 14]

The method according to any one of Supplementary Notes 1 to 13,comprising excluding a combination other than positive and negativeexamples corresponding to a combination regarded as noise from theextracted combinations other than positive and negative examples, fromextraction.

[Supplementary Note 15]

The method according to any one of Supplementary Notes 1 to 13,comprising:

judging whether an inputted combination is a combination regarded asnoise based on a noise condition; and

the combinations other than positive and negative examples correspondingto the combination regarded as the noise are excluded from theextraction result.

[Supplementary Note 16]

The method according to any one of Supplementary Notes 1 to 15,

wherein at least one of generating and adding a combination to be apseudo positive example combination, generating and adding a combinationto be a pseudo negative example combination, and deleting a part of thepositive example combinations and the negative example combinations isexecuted to generate corrected positive example combinations andcorrected negative example combinations, and

wherein attribute data based on the corrected positive examplecombinations and the corrected negative example combinations isgenerated to learn the discriminant model.

[Supplementary Note 17]

The method according to Supplementary Note 16, wherein the correctedpositive example combinations and the corrected negative examplecombinations are generated so that imbalance between the positiveexample combinations and the negative example combinations is corrected.

[Supplementary Note 18]

The method according to Supplementary Note 16, wherein covariate shiftlearning is applied to generate the corrected positive examplecombinations and the corrected negative example combinations.

[Supplementary Note 19]

The method according to any one of Supplementary Notes 1 to 18, whereinthe positive example combinations are divided according to difference inthe degree of seriousness of disease as an adverse event, and adiscriminant model for each seriousness degree for discriminating apositive example combination and a discriminant model for discriminatinga negative example combination are used.

[Supplementary Note 20]

The method according to any one of Supplementary Notes 1 to 19,

wherein the medical information data is grouped by grouping drugs ordiseases based on a grouping condition to obtain grouped medicalinformation data, and

wherein the grouped medical information data is used to create theattribute data according to the grouped drugs or diseases.

[Supplementary Note 21]

A drug adverse event extraction apparatus for extracting a combinationof a drug and a disease corresponding to a drug adverse event, theapparatus comprising, on the assumption that combinations already knownas combinations indicating drug adverse events are regarded as positiveexample combinations, combinations already known as combinations notbeing drug adverse events are regarded as negative example combinations,and given combinations being neither positive example combinations nornegative example combinations are regarded as combinations other thanpositive and negative examples:

attribute creation means that generates, using medical information datathat includes time-series information about medical events for eachpatient stored in a storage device, attribute data for each of thepositive example combinations stored in the storage device, for each ofthe negative example combinations stored in the storage device and foreach of the combinations other than positive and negative examplesstored in the storage device, based on the time-series information aboutthe medical events, and stores the attribute data into the storagedevice;

learning means that learns a discriminant model by the attribute datacorresponding to the positive example combinations and the attributedata corresponding to the negative example combinations;

calculation means that inputs the attribute data corresponding to thecombinations other than positive and negative examples stored in thestorage device to the discriminant model to calculate scores; and

extraction means that applies an extraction condition to the scorecalculated for each of the combinations other than positive and negativeexamples to extract combinations other than positive and negativeexamples being suspected to be drug adverse events,

wherein the medical events for each patient include prescription of adrug for the patient and a disease observed in the patient, and

wherein the medical events for each patient further include at least oneof a medical act performed for the patient and an event showing that themedical act has been performed accompanying the medical act performedfor the patient.

[Supplementary Note 22]

The apparatus according to Supplementary Note 21, wherein the medicalevents include, for the patient, at least one of a drug newlyprescribed, a disease other than diseases specified by correspondingcombinations, hospitalization/non-hospitalization, request for medicalexpenses, and a hospital department the patient has seen a doctor.

[Supplementary Note 23]

The apparatus according to Supplementary Note 21 or 22, wherein theattribute data is data showing, for each of the combinations,characteristics of occurrence and non-occurrence of a medical event thatis at least one of a medical act performed for the patient and an eventshowing that the medical act has been performed, accompanying themedical act performed for the patient, at a time close to a time pointwhen a drug and disease of the combination co-occur on the same patienton the medical information data.

[Supplementary Note 24]

The apparatus according to Supplementary Note 21 or 22, wherein, as theattribute data, at least one of the following is used:

a pattern indicating, with a combination of a drug and a diseaseregarded as a combination targeted by attribute data generation in acase where the disease occurs within a predetermined first period afterprescription of the drug, occurrence/non-occurrence of a predeterminedkind of medical event during a second period with a predetermined lengththat includes a time point of the occurrence of the disease, in a timeseries;

an occurrence rate of a predetermined kind of medical event within aperiod from prescription of the drug until occurrence of a disease, witha combination of the drug and the disease regarded as a combinationtargeted by attribute data generation in the case where the diseaseoccurs within the first period;

a transition probability about, with a combination of the drug and adisease regarded as a combination targeted by attribute data generationin a case where the disease occurs within the first period, whether apredetermined kind of medical event occurs during a second period with apredetermined length that includes a time point of the occurrence of thedisease;

probability of, with a combination of the drug and a disease regarded asa combination targeted by attribute data generation in a case where thedisease occurs within the first period, one of hypotheses being rejectedwhen the hypothesis is correct, as a result of examination of thehypotheses, the hypotheses being a null hypothesis that there is not adifference in the rate of occurrence of a predetermined kind of medicalevent before and after a time point of the occurrence of the disease andan alternative hypothesis that there is a difference;

a numerical value indicating, for a drug and disease of each inputtedcombination, how abnormal each pattern included in a second medicalevent pattern set is as a pattern generated from a learned probabilitymodel, the numerical value being obtained by learning with theprobability model with the use of: a first medical event pattern set inwhich results of extracting patterns indicating order of occurrence andnon-occurrence of a predetermined kind of medical event within apredetermined third period before and after a time point of prescriptionof the drug for the same patient as a reference point are collected, andthe second medical event pattern set in which results of extractingpatterns indicating order of occurrence and non-occurrence of thepredetermined kind of medical event within the third period before andafter a time point of occurrence of the disease as a reference point ina case where the disease occurs within the first period after theprescription of the drug for the same patient are collected; and

a degree of, for the pattern attribute data about the drug and diseaseof each inputted combination, how much the pattern attribute data isdeviated in comparison with pattern attribute data of other inputtedcombinations, based on magnitude tendency of the value of each attributeitem in the pattern attribute data of each combination.

[Supplementary Note 25]

The apparatus according to Supplementary Note 23 or 24, wherein theattribute data includes, for the drug and disease of each inputtedcombination, data indicating which ICD10 code the disease belongs to.

[Supplementary Note 26]

The apparatus according to any one of Supplementary Notes 21 to 25,wherein said attribute creation means executes preprocessing forgenerating time-series information about a new medical event that is notdirectly included in the medical information data but is used forgeneration of the attribute data, from time-series information about theplurality of medical events included in the medical information data.

[Supplementary Note 27]

The apparatus according to any one of Supplementary Notes 21 to 16,further comprising extraction means that extracts the combinations fromthe medical information data, classifies the extracted combinations intothe positive example combinations and the negative example combinationsbased on a dictionary, and stores the positive example combinations andthe negative example combinations into the storage device.

[Supplementary Note 28]

The apparatus according to Supplementary Note 27, wherein diseases to beignored are further described in the dictionary; and said combinationextraction means classifies the extracted combinations that include adisease other than the diseases to be ignored into the positive examplecombinations and the negative example combinations.

[Supplementary Note 29]

The apparatus according to any one of Supplementary Notes 21 to 28,wherein the discriminant model is provided for each kind of drug, andsaid calculation means calculates the adverse event score for the kindof drug.

[Supplementary Note 30]

The apparatus according to any one of Supplementary Notes 21 to 28,wherein a different discriminant model according to whether frequency ofeach combination in the medical information data is high or low isprovided, and said calculation means calculates the adverse event scoreaccording to whether the frequency is high or low.

[Supplementary Note 31]

The apparatus according to any one of Supplementary Notes 21 to 30,wherein said extraction means outputs an extraction result fromcombinations strongly suspected to be drug adverse events based on thescores.

[Supplementary Note 32]

The apparatus according to Supplementary Note 31, wherein saidextraction means outputs the extraction result according to kinds ofdrugs.

[Supplementary Note 33]

The apparatus according to any one of Supplementary Notes 21 to 30,comprising grouping means that performs grouping for the result obtainedby said extraction means.

[Supplementary Note 34]

The apparatus according to any one of Supplementary Notes 21 to 33,wherein said extraction means excludes the combinations other thanpositive and negative examples corresponding to the combination regardedas noise from extraction.

[Supplementary Note 35]

The apparatus according to any one of Supplementary Notes 21 to 33,further comprising noise extraction means that judges whether aninputted combination is a combination regarded as noise based on a noisecondition,

wherein said extraction means excludes the combinations other thanpositive and negative examples corresponding to the combination regardedas noise from extraction.

[Supplementary Note 36]

The apparatus according to any one of Supplementary Notes 21 to 35,further comprising correction means that executes at least one ofgenerating and adding a combination to be a pseudo positive examplecombination, generating and adding a combination to be a pseudo negativeexample combination, and deleting a part of the positive examplecombinations and the negative example combinations to generate correctedpositive example combinations and corrected negative examplecombinations,

wherein said attribute creation means creates attribute datacorresponding to the corrected positive example combinations andattribute data based on the corrected negative example combinations, and

wherein said learning means learns a discriminant model by the attributedata corresponding to the corrected positive example combinations andthe attribute data corresponding to the corrected negative examplecombinations.

[Supplementary Note 37]

The apparatus according to Supplementary Note 36, wherein saidcorrection means generates the corrected positive example combinationsand the corrected negative example combinations so that imbalancebetween the positive example combinations and the negative examplecombinations is corrected.

[Supplementary Note 38]

The apparatus according to Supplementary Note 36, wherein saidcorrection means comprises covariate shift means that applies covariateshift learning to generate the corrected positive example combinationsand the corrected negative example combinations.

[Supplementary Note 39]

The apparatus according to any one of Supplementary Notes 31 to 38,wherein the positive example combinations are divided according todifference in the degree of seriousness of disease as an adverse event,and a discriminant model for each seriousness degree for discriminatinga positive example combination and a discriminant model fordiscriminating a negative example combination are used.

[Supplementary Note 40]

The apparatus according to any one of Supplementary Notes 31 to 39,comprising prior grouping means that groups the medical information databy grouping drugs or diseases based on a grouping condition to obtaingrouped medical information data, and stores the grouped medicalinformation data into the storage device,

wherein said attribute creation means uses the grouped medicalinformation data to create the attribute data according to the groupeddrugs or diseases.

[Supplementary Note 41]

A program for causing a computer to which a combination of a drug and adisease is inputted to, on the assumption that combinations alreadyknown as combinations indicating drug adverse events are regarded aspositive example combinations, combinations already known ascombinations not being drug adverse events are regarded as negativeexample combinations, and given combinations being neither positiveexample combinations nor negative example combinations are regarded ascombinations other than positive and negative examples, function as:

attribute creation means that generates, using medical information datathat includes time-series information about medical events for eachpatient, attribute data for each of the positive example combinations,for each of the negative example combinations and for each of thecombinations other than positive and negative examples, based on thetime-series information about the medical events;

learning means that learns a discriminant model by the attribute datacorresponding to the positive example combinations and the attributedata corresponding to the negative example combinations;

calculation means that inputs the attribute data corresponding to thecombinations other than positive and negative examples to thediscriminant model to calculate scores; and

extraction means that applies an extraction condition to the scorecalculated for each of the combinations other than positive and negativeexamples to extract combinations other than positive and negativeexamples being suspected to be drug adverse events,

wherein the medical events for each patient include prescription of adrug for the patient and a disease observed in the patient, and

the medical events for each patient further include at least one of amedical act performed for the patient and an event showing that themedical act has been performed accompanying the medical act performedfor the patient.

REFERENCE SIGNS LIST

-   11 processor;-   12 storage device;-   13 communication interface (I/F) unit;-   14 operation input unit;-   15 screen display unit;-   21 input unit;-   22 attribute data creation unit;-   23 discriminant model learning unit;-   24 adverse event score calculation unit;-   25 extraction unit;-   26 combination extraction unit;-   27,28 grouping unit;-   31 noise extraction unit;-   32 imbalance correction unit;-   33 covariate shift processing unit.

The invention claimed is:
 1. A drug adverse event extraction method ofextracting a combination of a drug and a disease corresponding to a drugadverse event, assuming that combinations already known as combinationsindicating drug adverse events are regarded as positive examplecombinations, combinations already known as combinations not being drugadverse events are regarded as negative example combinations, and givencombinations being neither positive example combinations nor negativeexample combinations are regarded as combinations other than positiveand negative examples, the method comprising: retrieving a plurality ofthe positive example combinations indicating the drug adverse events fora plurality of diseases and a plurality of drugs stored in a storagedevice, each positive example combination being one of the plurality ofdiseases and one of the plurality of drugs associated together;retrieving a plurality of the negative example combinations notindicating the drug adverse events stored in the storage device, eachnegative example combination being one of the plurality of diseases andone of the plurality of drugs associated together; retrieving aplurality of the combinations other than positive and negative examples,each combination other than the positive and negative examples being oneof the plurality of diseases and one of the plurality of drugsassociated together; generating, using medical information data thatincludes time-series information about medical events for each patient,attribute data for each of the retrieved positive example combinations,for each of the retrieved negative example combinations, and for each ofthe retrieved combinations other than positive and negative examples,based on the time-series information about the medical events; readingthe attribute data for each of the positive example combinations and theattribute data for each of the negative example combinations, andmachine-learning a discriminant model installed in a computer based onthe attribute data corresponding to the read positive examplecombinations and the attribute data corresponding to the read negativeexample combinations; inputting the attribute data corresponding to thecombinations other than positive and negative examples to thediscriminant model and calculating scores; applying an extractioncondition to the score calculated for each of the combinations otherthan positive and negative examples to extract combinations other thanpositive and negative examples being suspected to be drug adverseevents; and outputting the extracted combinations suspected to be drugadverse events to one of a display or outside via a communicationinterface, wherein the medical events for each patient includeprescription of one of the drugs for the patient and one of the diseasesobserved in the patient, and wherein the medical events for each patientfurther include at least one of a medical act performed for the patientand an event showing that the medical act has been performedaccompanying the medical act performed for the patient.
 2. The methodaccording to claim 1, wherein the medical events include, for thepatient, at least one of a newly-prescribed drug, a disease other thandiseases specified by corresponding combinations,hospitalization/non-hospitalization, a request for medical expenses, anda hospital department at which the patient has seen a doctor.
 3. Themethod according to claim 1, wherein the attribute data is data showing,for each of the combinations, characteristics of occurrence andnon-occurrence of a medical event that is at least one of a medical actperformed for the patient and an event showing that the medical act hasbeen performed, accompanying the medical act performed for the patient,at a time close to a time point when a drug and disease of thecombination co-occur on the same patient on the medical informationdata.
 4. The method according to claim 3, wherein, as the attributedata, at least one of the following is used: a pattern indicating, witha combination of a drug and a disease regarded as a combination targetedby attribute data generation in a case where the disease occurs within apredetermined first period after prescription of the drug,occurrence/non-occurrence of a predetermined kind of medical eventduring a second period with a predetermined length that includes a timepoint of the occurrence of the disease, in a time series, an occurrencerate of a predetermined kind of medical event within a period fromprescription of the drug until occurrence of a disease, with acombination of the drug and the disease regarded as a combinationtargeted by attribute data generation in the case where the diseaseoccurs within the first period, a transition probability about, with acombination of the drug and a disease regarded as a combination targetedby attribute data generation in the case where the disease occurs withinthe first period, whether a predetermined kind of medical event occursduring a second period with a predetermined length that includes a timepoint of the occurrence of the disease, a probability of, with acombination of the drug and a disease regarded as a combination targetedby attribute data generation in the case where the disease occurs withinthe first period, one of hypotheses being rejected when the hypothesisis correct, as a result of examination of the hypotheses, the hypothesesbeing a null hypothesis that there is not a difference in the rate ofoccurrence of a predetermined kind of medical event before and after atime point of the occurrence of the disease and an alternativehypothesis that there is a difference, a numerical value indicating, fora drug and a disease of each inputted combination, how abnormal eachpattern included in a second medical event pattern set is as a patterngenerated from a learned probability model, the numerical value beingobtained by learning the probability model with the use of: a firstmedical event pattern set in which results of extracting patternsindicating order of occurrence and non-occurrence of a predeterminedkind of medical event within a predetermined third period before andafter a time point of prescription of the drug for the same patient as areference point are collected, and the second medical event pattern setin which results of extracting patterns indicating order of occurrenceand non-occurrence of the predetermined kind of medical event within thethird period before and after a time point of occurrence of the diseaseas a reference point in the case where the disease occurs within thefirst period after the prescription of the drug for the same patient arecollected, and a degree of, for the pattern attribute data about thedrug and disease of each inputted combination, how much the patternattribute data is deviated in comparison with pattern attribute data ofother inputted combinations, based on magnitude tendency of a value ofeach attribute item in the pattern attribute data of each combination.5. The method according to claim 3, wherein the attribute data includes,for the drug and disease of each inputted combination, data indicatingwhich ICD10 code the disease belongs to.
 6. The method according toclaim 1, further comprising executing preprocessing for generatingtime-series information about a new medical event that is not directlyincluded in the medical information data but is used for generation ofthe attribute data, from time-series information about the plurality ofmedical events included in the medical information data.
 7. The methodaccording to claim 1, further comprising extracting the combinationsfrom the medical information data and classifying the extractedcombinations into the positive example combinations and the negativeexample combinations based on a dictionary, before generating theattribute data.
 8. The method according to claim 1, wherein thediscriminant model for each kind of drug is used to calculate theadverse event score for the kind of drug.
 9. The method according toclaim 1, wherein a different discriminant model is used according towhether frequency of each combination in the medical information data ishigh or low to calculate the adverse event score according to whetherthe frequency is high or low.
 10. The method according to claim 1,further comprising grouping the extracted combinations other thanpositive and negative examples.
 11. The method according to claim 1,wherein at least one of generating and adding a combination to be apseudo positive example combination, generating and adding a combinationto be a pseudo negative example combination, and deleting a part of thepositive example combinations and the negative example combinations isexecuted to generate corrected positive example combinations andcorrected negative example combinations, and wherein attribute databased on the corrected positive example combinations and the correctednegative example combinations is generated to learn the discriminantmodel.
 12. The method according to claim 1, wherein the positive examplecombinations are divided according to difference in the degree ofseriousness of disease as an adverse event, and a discriminant model foreach seriousness degree for discriminating a positive examplecombination and a discriminant model for discriminating a negativeexample combination are used.
 13. The method according to claim 1,wherein the medical information data is grouped by grouping drugs ordiseases based on a grouping condition to obtain grouped medicalinformation data, and wherein the grouped medical information data isused to create the attribute data according to the grouped drugs ordiseases.
 14. A drug adverse event extraction apparatus for extracting acombination of a drug and a disease corresponding to a drug adverseevent, assuming that combinations already known as combinationsindicating drug adverse events are regarded as positive examplecombinations, combinations already known as combinations not being drugadverse events are regarded as negative example combinations, and givencombinations being neither positive example combinations nor negativeexample combinations are regarded as combinations other than positiveand negative examples, the apparatus comprising: an input unitconfigured to retrieve a plurality of the positive example combinationsindicating the drug adverse events for a plurality of diseases and aplurality of drugs stored in a storage device, each positive examplecombination being one of the plurality of diseases and one of theplurality of drugs associated together, retrieve a plurality of thenegative example combinations not indicating the drug adverse eventsstored in the storage device, each negative example combination beingone of the plurality of diseases and one of the plurality of drugsassociated together, and retrieve a plurality of the combinations otherthan positive and negative examples, each combination other than thepositive and negative examples being one of the plurality of diseasesand one of the plurality of drugs associated together; an attribute datacreation unit that generates, using medical information data thatincludes time-series information about medical events for each patientstored in the storage device, attribute data for each of the retrievedpositive example combinations stored in the storage device, for each ofthe retrieved negative example combinations stored in the storage deviceand for each of the retrieved combinations other than positive andnegative examples stored in the storage device, based on the time-seriesinformation about the medical events, and stores the attribute data intothe storage device; a machine-learning unit that reads the attributedata for each of the positive example combinations and the attributedata for each of the negative example combinations, and learns adiscriminant model installed in a computer based on the attribute datacorresponding to the positive example combinations and the attributedata corresponding to the negative example combinations; a calculationunit that inputs the attribute data corresponding to the combinationsother than positive and negative examples stored in the storage deviceto the discriminant model and calculates scores; an extraction unit thatapplies an extraction condition to the score calculated for each of thecombinations other than positive and negative examples to extractcombinations other than positive and negative examples being suspectedto be drug adverse events; and one of a display configured to displaythe extracted combinations suspected to be drug adverse events, and acommunication interface configured to output the extracted combinationssuspected to be drug adverse events to outside, wherein the medicalevents for each patient include prescription of a drug for the patientand a disease observed in the patient, and wherein the medical eventsfor each patient further include at least one of a medical act performedfor the patient and an event showing that the medical act has beenperformed accompanying the medical act performed for the patient. 15.The apparatus according to claim 14, wherein the medical events include,for the patient, at least one of a newly-prescribed drug, a diseaseother than diseases specified by corresponding combinations,hospitalization/non-hospitalization, a request for medical expenses, anda hospital department at which the patient has seen a doctor.
 16. Theapparatus according to claim 14, wherein the attribute data is datashowing, for each of the combinations, characteristics of occurrence andnon-occurrence of a medical event that is at least one of a medical actperformed for the patient and an event showing that the medical act hasbeen performed, accompanying the medical act performed for the patient,at a time close to a time point when a drug and disease of thecombination co-occur on the same patient on the medical informationdata.
 17. The apparatus according to claim 14, further comprising acombination extraction unit that extracts the combinations from themedical information data, classifies the extracted combinations into thepositive example combinations and the negative example combinationsbased on a dictionary, and stores the positive example combinations andthe negative example combinations into the storage device.
 18. Theapparatus according to claim 14, further comprising a grouping unit thatperforms grouping for a result obtained by said extraction unit.
 19. Theapparatus according to claim 14, further comprising a correction unitthat executes at least one of generating and adding a combination to bea pseudo positive example combination, generating and adding acombination to be a pseudo negative example combination, and deleting apart of the positive example combinations and the negative examplecombinations to generate corrected positive example combinations andcorrected negative example combinations, wherein said attribute creationunit creates attribute data corresponding to the corrected positiveexample combinations and attribute data based on the corrected negativeexample combinations, and wherein said learning unit learns adiscriminant model by the attribute data corresponding to the correctedpositive example combinations and the attribute data corresponding tothe corrected negative example combinations.
 20. The apparatus accordingto claim 14, further comprising a prior grouping unit that groups themedical information data by grouping drugs or diseases based on agrouping condition to obtain grouped medical information data, andstores the grouped medical information data into the storage device,wherein said attribute creation unit uses the grouped medicalinformation data to create the attribute data according to the groupeddrugs or diseases.